Feature Selection Regression Research Articles

Predicting MCI to AD Conversation Using Integrated sMRI and rs-fMRI: Machine Learning and Graph Theory Approach.

BackgroundGraph theory and machine learning have been shown to be effective ways of classifying different stages of Alzheimer’s disease (AD). Most previous studies have only focused on inter-subject classification with single-mode neuroimaging data. However, whether this classification can truly reflect the changes in the structure and function of the brain region in disease progression remains unverified. In the current study, we aimed to evaluate the classification framework, which combines structural Magnetic Resonance Imaging (sMRI) and resting-state functional Magnetic Resonance Imaging (rs-fMRI) metrics, to distinguish mild cognitive impairment non-converters (MCInc)/AD from MCI converters (MCIc) by using graph theory and machine learning.MethodsWith the intra-subject (MCInc vs. MCIc) and inter-subject (MCIc vs. AD) design, we employed cortical thickness features, structural brain network features, and sub-frequency (full-band, slow-4, slow-5) functional brain network features for classification. Three feature selection methods [random subset feature selection algorithm (RSFS), minimal redundancy maximal relevance (mRMR), and sparse linear regression feature selection algorithm based on stationary selection (SS-LR)] were used respectively to select discriminative features in the iterative combinations of MRI and network measures. Then support vector machine (SVM) classifier with nested cross-validation was employed for classification. We also compared the performance of multiple classifiers (Random Forest, K-nearest neighbor, Adaboost, SVM) and verified the reliability of our results by upsampling.ResultsWe found that in the classifications of MCIc vs. MCInc, and MCIc vs. AD, the proposed RSFS algorithm achieved the best accuracies (84.71, 89.80%) than the other algorithms. And the high-sensitivity brain regions found with the two classification groups were inconsistent. Specifically, in MCIc vs. MCInc, the high-sensitivity brain regions associated with both structural and functional features included frontal, temporal, caudate, entorhinal, parahippocampal, and calcarine fissure and surrounding cortex. While in MCIc vs. AD, the high-sensitivity brain regions associated only with functional features included frontal, temporal, thalamus, olfactory, and angular.ConclusionsThese results suggest that our proposed method could effectively predict the conversion of MCI to AD, and the inconsistency of specific brain regions provides a novel insight for clinical AD diagnosis.

Deep neural network features fusion and selection based on PLS regression with an application for crops diseases classification

Objective:The plants diseases affect both the production and quality of food in the agriculture sector. Computer vision techniques can contribute significantly by detecting the plant’s diseases at very early stages with more accuracy. Method:In this work, we proposed an automated crop disease recognition system using partial least squares (PLS) regression for feature selection from an extracted deep feature set. The presented framework incorporates three primary phases: First, the deep features are extracted using a pre-trained Visual Geometry Group (VGG19) convolutional neural networks (CNN) model; Second, a PLS-based parallel fusion method combines the features extracted from the fully connected layers 6 and 7; Third, the best features are selected using a PLS projection method. The most discriminant features are finally plugged into the ensemble baggage tree classifier for final recognition. Results:Three different crops (tomato, corn and potato) are selected from the Plant Village dataset for the algorithm’s evaluation. The average accuracy achieved using the proposed method is approximately 90.1%. Conclusion:The proposed PLS based fusion and selection methods not only improve recognition accuracy but also reduce computational time. Further, based on the achieved results, we are confident that the proposed plan will work even under light variations and texture constraints.

Genetic algorithm with logistic regression feature selection for Alzheimer’s disease classification

Cognitive impairment must be diagnosed in Alzheimer’s disease as early as possible. Early diagnosis allows the person to receive effective treatment benefits apart from helping him or her to remain independent longer. In this paper, different feature selection techniques are utilized with different classifiers in the classification of this chronic disease as normal control (NC), mild cognitive impairment (MCI) and Alzheimer’s disease (AD) based on the MRI images of ADNI dataset. Dimensionality reduction plays a major role in improving classification performance when there are fewer records with high dimensions. After different trials to select the ample features, support vector machine (SVM) with radial basis function kernel is found to produce better results with 96.82%, 89.39% and 90.40% accuracy for binary classification of NC/AD, NC/MCI and MCI/AD, respectively, with repeated tenfold stratified cross-validation. Combining mini-mental state examination (MMSE) score to the MRI data, there has been an improvement of 2.7% in the MCI/AD classification, but it does not have much influence in the NC/AD and NC/MCI classification.

A Robust Linear Regression Feature Selection Method for Data Sets with Unknown Noise

The linear regression model is simple in form and easy to estimate; nevertheless, irrelevant features will raise the difficulty of its tasks. Feature selection is generally adopted to improve the model performance. Unfortunately, traditional regression feature selection methods may not work for data with noise or outliers. Although some robust methods for certain specific error distributions have been proposed, they may not perform well because the distribution of representation error is often unknown for real data. This paper proposes a regression feature selection method for unknown noise named Mixture of Gaussians LASSO (MoG-LASSO), in which feature selection and model training will be achieved simultaneously. MoG is adopted to model unknown noises, and M-estimation is used to acquire the weighted squared error loss. By alternatively and iteratively updating the regression coefficient and parameters of MoG, the influence of unknown noise can be reduced effectively. Furthermore, MoG-LASSO achieves feature selection by the $L_{1}$ regularization term, which can further improve the performance of the model. Experimental results on artificial data and benchmark data sets demonstrate that MoG-LASSO has better robustness and sparsity for data sets with irrelevant features. Additionally, experimental results on face recognition databases show the performance advantage of MoG-LASSO over state-of-the-art methods in the presence of illumination variations.

Detection and classification of breast cancer using logistic regression feature selection and GMDH classifier.

Breast cancer is the most common cancer among women such that the existence of a precise and reliable system for the diagnosis of benign or malignant tumors is critical. Nowadays, using the results of Fine Needle Aspiration (FNA) cytology and machine learning techniques, detection and early diagnosis of this cancer can be done with greater accuracy. In this paper, we propose a method consisting of two steps: in the first step, to eliminate the less important features, logistic regression has been used. In the second step, the Group Method Data Handling (GMDH) neural network is used for the diagnosis of benign and malignant samples. To evaluate the performance of the proposed method, three datasets WBCD, WDBC and WPBC are investigated with metrics: precision, the Area Under the ROC (AUC), true positive rate, false positive rate, accuracy and F-criteria. Simulation results show that the proposed method reaches a precision of 99.4% for WBCD, 99.6% for WDBC and a precision of 96.9% for WPBC dataset.

Development and Validation of a Five-immune Gene Pair Signature in Endometrial Carcinoma.

Endometrial cancer (EC) is a common gynecological malignancy worldwide. Immunity is closely related to the occurrence and prognosis of EC. At the same time, immune-related genes have great potential as prognostic markers in many types of cancer. Therefore, we attempt to develop immune-related gene markers to enhance prognosis prediction of EC. 542 samples of EC gene expression data and clinical follow-up information were downloaded from The Cancer Genome Atlas (TCGA). The samples were randomly divided into two groups, one group as a training set (N=271), and one set as a validation set. (N=271). In the training set, the gene pairs were established based on the relative expression levels of 271 immune genes, and the prognosis-related gene pairs were screened. The lasso was used to select the features, and finally, the robust biomarkers were screened. Finally, the prognostic model of the immune gene pair was established and verified by the validation data set. 10030 immune gene pair (IRGPs) were obtained, and univariate survival analysis was used to identify 1809 prognostic-related IRGPs (p<0.05). 5-IRGPs were obtained by lasso regression feature selection, and multivariate regression was used to establish 5-IRGPs signature, 5-IRGPs signature is an independent prognostic factor for EC patients, and could be risk stratified in patients with TCGA datasets, age, ethnicity, stage, and histological classification (p<0.05). The mean AUC of survival in both the training set and the validation set was greater than 0.7, indicating that 5-IRGPs signature has superior classification performance in patients with EC. In addition, 5-IRGPs have the highest average C index (0.795) compared to the prognostic characteristics of the three endometrial cancers reported in the past and Stage and Age. This study constructed a 5-IRGPs signature as a novel prognostic marker for predicting survival in patients with EC.

Abstract WMP101: Prediction of Clinical Outcome in Supratentorial Intracerebral Hemorrhage: Application of Baseline Ct Scan Radiomics Feature Extraction and Machine Learning Classifiers

Aim: Radiomics refers to automatic extraction of numerous quantitative features from medical images to supplement visual assessment. Machine-learning algorithms provide a suitable statistical methodology for devising predictive classifiers based on large radiomics datasets. We aimed to predict intracerebral hemorrhage (ICH) outcome by applying machine-learning classifiers to both clinical data and hematoma radiomics features. Methods: Patients enrolled in the Yale Longitudinal Study of ICH were included if they had (1) spontaneous supratentorial ICH, (2) baseline CT scan, (3) known admission Glasgow Coma Scale (GCS), and (4) 3-month modified Rankin Scale (mRS). A total of 1134 radiomics features related to the intensity, shape, texture, and waveform were extracted from manually segmented ICH lesions on baseline CT. Clinical variables were patients’ age, gender, GCS, presence of intraventricular hemorrhage, and thalamic ICH. We calculated the averaged receiver operating characteristics (ROC) area under curve (AUC) in outcome prediction among 100 repeats of 5-fold cross-validation (x500 iterations) for different combinations of feature selection and machine-learning algorithms. Results: A total of 119 ICH patients were included, of whom 60 had poor outcome (mRS ≥4). Among different combinations, lasso regression feature selection and partial least square (PLS) classification model yielded the highest accuracy in outcome prediction (Figure), with an averaged (95% confidence interval) ROC AUC of 0.86 (0.83 - 0.89) using clinical variables “only”, versus 0.92 (0.89 - 0.95) using combination of clinical variables and 54 radiomics features selected by lasso regression. Among radiomics features selected by lasso regression, ICH lesion flatness had the highest variable importance and was the only shape feature selected. Conclusion: Addition of ICH lesion radiomics to clinical variables using machine-learning models can improve outcome prediction.

Surface Heterogeneity-Involved Estimation of Sample Size for Accuracy Assessment of Land Cover Product from Satellite Imagery.

Sample size estimation is a key issue for validating land cover products derived from satellite images. Based on the fact that present sample size estimation methods account for the characteristics of the Earth’s subsurface, this study developed a model for estimating sample size by considering the scale effect and surface heterogeneity. First, we introduced a watershed with different areas to indicate the scale effect on the sample size. Then, by employing an all-subsets regression feature selection method, three landscape indicators describing the aggregation and diversity of the land cover patches were selected (from 14 indicators) as the main factors for indicating the surface heterogeneity. Finally, we developed a multi-level linear model for sample size estimation using explanatory variables, including the estimated sample size (n) calculated from the traditional statistical model, size of the test region, and three landscape indicators. As reference data for developing this model, we employed a case study in the Jiangxi Province using a 30 m spatial resolution global land cover product (Globeland30) from 2010 as a classified map, and national 30 m land use/cover change (LUCC) data from 2010 in China. The results showed that the adjusted square coefficient of R2 is 0.79, indicating that the joint explanatory ability of all predictive variables in the model to the sample size is 79%. This means that the predictability of this model is at a good level. By comparing the sample size NS obtained by the developed multi-level linear model and n as calculated from the statistics model, we find that NS is much smaller than n, which mainly contributes to the concerns regarding surface heterogeneity in this study. The validity of the established model is tested and is proven as effective in the Anhui Province. This indicates that the estimated sample size from considering the scale effect and spatial heterogeneity in this study achieved the same accuracy as that calculated from a probability statistical model, while simultaneously saving more time, labour, and money in the accuracy assessment of a land cover dataset.

Classification of Early and Late Mild Cognitive Impairment Using Functional Brain Network of Resting-State fMRI.

Using the Pearson correlation coefficient to constructing functional brain network has been evidenced to be an effective means to diagnose different stages of mild cognitive impairment (MCI) disease. In this study, we investigated the efficacy of a classification framework to distinguish early mild cognitive impairment (EMCI) from late mild cognitive impairment (LMCI) by using the effective features derived from functional brain network of three frequency bands (full-band: 0.01–0.08 Hz; slow-4: 0.027–0.08 Hz; slow-5: 0.01–0.027 Hz) at Rest. Graphic theory was performed to calculate and analyze the relationship between changes in network connectivity. Subsequently, three different algorithms [minimal redundancy maximal relevance (mRMR), sparse linear regression feature selection algorithm based on stationary selection (SS-LR), and Fisher Score (FS)] were applied to select the features of network attributes, respectively. Finally, we used the support vector machine (SVM) with nested cross validation to classify the samples into two categories to obtain unbiased results. Our results showed that the global efficiency, the local efficiency, and the average clustering coefficient were significantly higher in the slow-5 band for the LMCI–EMCI comparison, while the characteristic path length was significantly longer under most threshold values. The classification results showed that the features selected by the mRMR algorithm have higher classification performance than those selected by the SS-LR and FS algorithms. The classification results obtained by using mRMR algorithm in slow-5 band are the best, with 83.87% accuracy (ACC), 86.21% sensitivity (SEN), 81.21% specificity (SPE), and the area under receiver operating characteristic curve (AUC) of 0.905. The present results suggest that the method we proposed could effectively help diagnose MCI disease in clinic and predict its conversion to Alzheimer’s disease at an early stage.

An Adaptive Regression Feature Selection Method for Datasets with Outliers

An Adaptive Regression Feature Selection Method for Datasets with Outliers

Abstract P5-11-10: Biomarkers associated with resistance or response to CDK4/6 treatment in patients with metastatic hormone-receptive positive breast cancer

Abstract Background: CDK4/6 inhibitor (CDKi) drugs are the current standard of care for treatment of first and second-line hormone-receptor positive/HER2 negative (HR+/HER2-) metastatic breast cancers. Numerous research efforts have commenced to understand biomarkers of response and resistance. To date, no biomarker of response has been identified. Treatment induced RB1 mutations were noted as mechanism of resistance to palbociclib and fulvestrant in about 5% of patients treated on PALOMA3 trials, whereas PI3K and ESR1 mutations emerged as potential resistance to the anti-hormonal backbone1. Additionally, FGFR1 amplification has been suggested as a resistance pathway to fulvestrant and ribociclib2. A better understanding of the molecular landscape of CDK4/6 treatment is needed. Utilizing next-gen sequencing (NGS), chromogenic in situ hybridization (CISH) and immunohistochemistry (IHC) data from HR+/HER2- patients treated at the University of Tennessee West Cancer Center, we attempted to retrospectively identify a molecular signature of resistance or response as measured by PFS. Methods: We analyzed 115 breast cancer patients who received CDKi treatment and 30 matched controls not exposed to CDKi and underwent comprehensive molecular profiling (Caris Life Sciences, Phoenix, AZ). A Cox proportional hazards model was built using genetic test as predictors and progression free survival (PFS) time as response. Only alterations with known pathogenic potential were considered aberrant. The R package glmnet was used to perform regularized lasso regression for feature selection on the entire data set. Important features were then used to construct Kaplan-Meier curves and perform a log-rank test for difference in PFS times. Results: Here we report the analysis for 2 known pathogenic biomarkers, ESR1 and TP53 based on PFS for patients who test positive versus negative. The median PFS for all patients was 234 days. Patients who harbored ESR1 mutations had reduced PFS of 180 days compared to 237 days. Patients who had P53 mutations had shorter PFS of 201 days compared to 240 days. When both groups with positive mutations were combined, the median PFS was 191 days compared to 276 days for patients without either ESR1 or P53 mutation with a p-value of 0.011. Further analysis using 4 way Kaplan Meier Curves for controls versus treated, altered versus non-altered genes, or PDL-1 expression is ongoing and will be presented at the conference. Conclusion: This data further support, in a real world model, the poor predictive value of ESR1 and P53 mutations on clinical outcome. Because no testing data was used, additional validation will be necessary to confirm the findings from this analysis.

Wide spectrum feature selection (WiSe) for regression model building

Developing predictive models from industrial datasets implies the consideration of many possible predictor variables (features). Using all available features for data-driven modelling is not recommended, as most of them are expected to be irrelevant and their inclusion in the model may compromise robustness and accuracy. In this work, we present, test and compare a new two-stage feature selection method called wide spectrum feature selection for regression (WiSe). In the first stage, a combination of efficient bivariate filters analyzes linear and non-linear association patterns between predictors and responses, screening out clearly noisy features. In the second stage, the reduced set of retained features is subject to further selection in the scope of the predictive methods considered, optimizing their predictive performance. Three simulated datasets and an industrial case illustrate the effectiveness and benefits of applying WiSe to support model development in a wide range of high-dimensional regression problems.

Acute trajectories of neural activation predict remission to pharmacotherapy in late-life depression.

Pharmacological treatment of major depressive disorder (MDD) typically involves a lengthy trial and error process to identify an effective intervention. This lengthy period prolongs suffering and worsens all-cause mortality, including from suicide, and is typically longer in late-life depression (LLD). Our group has recently demonstrated that during an open-label venlafaxine (serotonin-norepinephrine reuptake inhibitor) trial, significant changes in functional resting state connectivity occurred following a single dose of treatment, which persisted until the end of the trial. In this work, we propose an analysis framework to translate these perturbations in functional networks into predictors of clinical remission. Participants with LLD (N = 49) completed 12-weeks of treatment with venlafaxine and underwent functional magnetic resonance imaging (fMRI) at baseline and a day following a single dose of venlafaxine. Data was collected at rest as well as during an emotion reactivity task and an emotion regulation task. Remission was defined as a Montgomery-Asberg Depression Rating Scale (MADRS) ≤10 for two weeks. We computed eigenvector centrality (whole brain connectivity) and activation during the emotion regulation and emotion reactivity tasks. We employed principal components analysis, Tikhonov-regularized logistic classification, and least angle regression feature selection to predict remission by the end of the 12-week trial. We utilized ten-fold cross-validation and Receiver Operator Curves (ROC) curve analysis. To determine task-region pairs that significantly contributed to the algorithm's ability to predict remission, we used permutation testing. Using the fMRI data at both baseline and after the first dose of treatment yielded a sensitivity of 72% and a specificity of 68% (AUC = 0.77), a 15% increase in accuracy over baseline MADRS. In general, the accuracy at baseline was further improved by using the change in activation following a single dose. Activation of the frontal cortex, hippocampus, parahippocampus, caudate, thalamus, medial temporal cortex, middle cingulate, and visual cortex predicted treatment remission. Acute, dynamic trajectories of functional imaging metrics in response to a pharmacological intervention are a valuable tool for predicting treatment response in late-life depression and elucidating the mechanism of pharmacological therapies in the context of the brain's functional architecture.

The randomized information coefficient: assessing dependencies in noisy data

When differentiating between strong and weak relationships using information theoretic measures, the variance plays an important role: the higher the variance, the lower the chance to correctly rank the relationships. We propose the randomized information coefficient (RIC), a mutual information based measure with low variance, to quantify the dependency between two sets of numerical variables. We first formally establish the importance of achieving low variance when comparing relationships using the mutual information estimated with grids. Second, we experimentally demonstrate the effectiveness of RIC for (i) detecting noisy dependencies and (ii) ranking dependencies for the applications of genetic network inference and feature selection for regression. Across these tasks, RIC is very competitive over other 16 state-of-the-art measures. Other prominent features of RIC include its simplicity and efficiency, making it a promising new method for dependency assessment.

Multi-objective evolutionary feature selection for online sales forecasting

Sales forecasting uses historical sales figures, in association with products characteristics and peculiarities, to predict short-term or long-term future performance in a business, and it can be used to derive sound financial and business plans. By using publicly available data, we build an accurate regression model for online sales forecasting obtained via a novel feature selection methodology composed by the application of the multi-objective evolutionary algorithm ENORA (Evolutionary NOn-dominated Radial slots based Algorithm) as search strategy in a wrapper method driven by the well-known regression model learner Random Forest. Our proposal integrates feature selection for regression, model evaluation, and decision making, in order to choose the most satisfactory model according to an a posteriori process in a multi-objective context. We test and compare the performances of ENORA as multi-objective evolutionary search strategy against a standard multi-objective evolutionary search strategy such as NSGA-II (Non-dominated Sorted Genetic Algorithm), against a classical backward search strategy such as RFE (Recursive Feature Elimination), and against the original data set.

L0-norm based structural sparse least square regression for feature selection

This paper presents a novel approach for feature selection with regard to the problem of structural sparse least square regression (SSLSR). Rather than employing the l1-norm regularization to control the sparsity, we directly work with sparse solutions via l0-norm regularization. In particular, we develop an effective greedy algorithm, where the forward and backward steps are combined adaptively, to resolve the SSLSR problem with the intractable lr,0-norm. On the one hand, features with the strongest correlation to classes are selected in the forward steps. On the other hand, redundant features which contribute little to the improvement of the objective function are removed in the backward steps. Furthermore, we provide solid theoretical analysis to prove the effectiveness of the proposed method. Experimental results on synthetic and real world data sets from different domains also demonstrate the superiority of the proposed method over the state-of-the-arts.

Machine learning methods in the computational biology of cancer

The objectives of this Perspective paper are to review some recent advances in sparse feature selection for regression and classification, as well as compressed sensing, and to discuss how these might be used to develop tools to advance personalized cancer therapy. As an illustration of the possibilities, a new algorithm for sparse regression is presented and is applied to predict the time to tumour recurrence in ovarian cancer. A new algorithm for sparse feature selection in classification problems is presented, and its validation in endometrial cancer is briefly discussed. Some open problems are also presented.

Is mutual information adequate for feature selection in regression?

Feature selection is an important preprocessing step for many high-dimensional regression problems. One of the most common strategies is to select a relevant feature subset based on the mutual information criterion. However, no connection has been established yet between the use of mutual information and a regression error criterion in the machine learning literature. This is obviously an important lack, since minimising such a criterion is eventually the objective one is interested in. This paper demonstrates that under some reasonable assumptions, features selected with the mutual information criterion are the ones minimising the mean squared error and the mean absolute error. On the contrary, it is also shown that the mutual information criterion can fail in selecting optimal features in some situations that we characterise. The theoretical developments presented in this work are expected to lead in practice to a critical and efficient use of the mutual information for feature selection.

Feature Selection in Computer-Aided Breast Cancer Diagnosis via Dynamic Contrast-Enhanced Magnetic Resonance Images

The accuracy of computer-aided diagnosis (CAD) for early detection and classification of breast cancer in dynamic contrast-enhanced magnetic resonance imaging (DCE-MRI) is dependent upon the features used by the CAD classifier. Here, we show that fast orthogonal search (FOS), which provides a more efficient iterative manner of computing stepwise regression feature selection, can select features with predictive value from a set of kinetic and texture candidate features computed from dynamic contrast-enhanced magnetic resonance images. FOS can in minutes search candidate feature sets of millions of terms, which may include cross-products of features up to second-, third- or fourth-order. This method is tested on a set of 83 DCE-MRI images, of which 20 are for cancerous and 63 for benign cases, using a leave-one-out trial. The features selected by FOS were used in a FOS predictor and nearest-neighbour predictor and had an area under the receiver operating curve (AUC) of 0.889 and 0.791 respectively. The FOS predictor AUC is significantly improved over the signal enhancement ratio predictor with an AUC of 0.706 (p = 0.0035 for the difference in the AUCs). Moreover, using FOS-selected features in a support vector machine increased the AUC over that resulting when the features were manually selected.

In silico prediction of free-radical chain transfer constants for some organic agents in styrene polymerization

In the present work, quantitative structure–reactivity relationship (QSRR) approaches were used to predict the chain transfer constant log C x of some organic compounds as chain transfer agents in free-radical polymerization of styrene. The energy of the lowest unoccupied molecular orbital, hydrogen-bonding-dependent hydrogen donor charged area, first-order Kier and Hall index, final heat of formation/number of atoms, count of H donor sites, and Min>(0.1) bond order of a C atom were selected as the most relevant variables from the pool of calculated descriptors by the stepwise multiple regression feature selection method. Then, an artificial neural network and multiple linear regressions were utilized to construct the nonlinear and linear QSRR models. The standard errors in the prediction of log C x by the linear regression model were 0.641, 0.964, and 0.843 and by the neural network model were 0.049, 0.076, and 0.090 for training, internal, and external test sets, respectively. The predictivity of the artificial neural network model was further examined by cross-validation methods, which produce a Q 2 value of 0.85. The results of this study revealed the applicability of QSRR approaches in prediction of the chain transfer constant by using an artificial neural network.