Guided Feature Selection Research Articles

Multimodality Fusion Method Based on Multiview Subspace Clustering for Pulmonary Embolism Diagnosis.

Pulmonary embolism (PE) is an important clinical disorder that will result in lung tissue damage or low blood oxygen levels, which need early diagnosis and timely treatment. While computed tomographic pulmonary angiography (CTPA) is the gold standard to diagnose PE, previous studies have verified the effectiveness of combing CTPA and EMR data in computer-aided PE detection or diagnosis. In this paper, we proposed a multimodality fusion method based on multi-view subspace clustering guided feature selection (MSCUFS). The extracted high-dimensional image and EMR features are firstly selected and fused by the MSCUFS, and then are feed into different machine learning models with different fusion strategy to construct the PE classifier. The experiment results showed that the joint fusion strategy with MSCUFS achieved best AUROC of 0.947, surpassing other early fusion and late fusion models. The comparison between single modality and multimodality also illustrated the effectiveness of the proposed method.

Prediction of New-Onset Diabetes after Pancreatectomy with Subspace Clustering based Multi-view Feature Selection.

The pancreas plays an important role in glucose metabolism, and developing diabetes or long-term glucose metabolism disturbance may be a prevalent sequela after pancreatectomy. Nevertheless, relative factors of new-onset diabetes after pancreatectomy stay unclear. Radiomics analysis is potential to identify image markers for disease prediction or prognosis. Meanwhile, combination of imaging and electronic medical record (EMR) showed superior performance than imaging or EMR alone in previous studies. One critical step is to identity predictors from high-dimensional features, and it is even more challenging to select and fuse imaging and EMR features. In this work, we develop a radiomics pipeline to assess postoperative new-onset diabetes risk of patients undergoing distal pancreatectomy. Specifically, we extract multiscale image features with 3D wavelet transformation, and include patients' characteristics, body composition and pancreas volume information as clinical features. Then, we propose a multi-view subspace clustering guided feature selection method (MSCUFS) for the selection and fusion of image and clinical features. Finally, a prediction model is constructed with classical machine learning classifier. Experimental results on an established distal pancreatectomy cohort showed that the SVM model with combined imaging and EMR features demonstrated good discrimination, with an AUC value of 0.824, which improved the model with image features alone by 0.037 AUC. Compared with state-of-the-art feature selection methods, the proposed MSCUFS has superior performance in fusing image and clinical features.

Distance-Preserving Embedding Adaptive Bipartite Graph Multi-View Learning with Application to Multi-Label Classification

Graph-based multi-view learning has attracted much attention due to the efficacy of fusing the information from different views. However, most of them exhibit high computational complexity. We propose an anchor-based bipartite graph embedding approach to accelerate the learning process. Specifically, different from existing anchor-based methods where anchors are obtained from key samples by clustering or weighted averaging strategies, in this article, the anchors are learned in a principled fashion which aims at constructing a distance-preserving embedding for each view from samples to their representations, whose elements are the weights of the edges linking corresponding samples and anchors. In addition, the consistency among different views can be explored by imposing a low-rank constraint on the concatenated embedding representations. We further design a concise yet effective feature collinearity guided feature selection scheme to learn tight multi-label classifiers. The objective function is optimized in an alternating optimization fashion. Both theoretical analysis and experimental results on different multi-label image datasets verify the effectiveness and efficiency of the proposed method.

UBayFS: An R Package for User Guided Feature Selection

UBayFS: An R Package for User Guided Feature Selection

Hybrid data‐driven hazard‐consistent drift models for SMRF

AbstractThe seismic design and assessment of steel moment resisting frames (SMRFs) rely heavily on drifts. It is unsurprising, therefore, that several simplified methods have been proposed to predict lateral deformations in SMRFs, ranging from the purely mechanics‐based to the wholly data‐driven, which aim to alleviate the structural engineer's burden of conducting detailed nonlinear analyses either as part of preliminary design iterations or during regional seismic assessments. While many of these methods have been incorporated in design codes or are commonly used in research, they all suffer from a lack of consideration of the causal link between the seismic hazard level and the ground‐motion suite used for their formulation. In this paper, we propose hybrid data‐driven models that preserve this critical relationship of hazard‐consistency. To this end, we assemble a large database of non‐linear response history analyses (NRHA) on 24 SMRFs of different structural characteristics. These structural models are subjected to 816 ground‐motion records whose occurrence rates and spectral shapes are selected to ensure the hazard consistency of our outputs. Two sites with different seismic hazards are examined to enable comparisons under different seismic demands. An initial examination of the resulting drift hazard curves allows us to re‐visit the influence of salient structural modelling assumptions such as plastic resistance, geometric configurations and joint deterioration modelling. This is followed by a machine learning (ML)‐guided feature selection process that considers structural and seismic parameters as well as key static response features, hence the hybrid nature of our models. New models for inter‐storey drift and roof displacements are then developed. A comparison with currently available formulations highlights the significant levels of overestimation associated with previously proposed non‐hazard consistent models.

Depth guided feature selection for RGBD salient object detection

Depth information can greatly benefit the saliency detection in RGBD images if they are utilized well. Prevalent methods generally directly fuse depth and RGB features in networks. However, due to the inherent inconsistent between RGB and depth information, the RGB features are easy to be interfered by the intrinsic noise existed in depth features, making the precise RGBD saliency detection still a challenge. In this paper, we propose a novel Depth Guided Feature Selection network (DGFSnet) that takes depth information as prior and dynamically selects the complementary RGB information for RGBD salient object detection. Specifically, DGFSnet first includes a Depth Weight Generation module (DWG) to learn a set of layer-specific weights from multi-scale depth features. Guiding by these learned weights, DGFSnet further devises a Weight-guided Feature Aggregation module (WFA) to assign them to their corresponding RGB layers for dynamically enhancing and selecting saliency-related RGB features. With two modules, DGFSnet is able to effectively integrate the multi-modality complementaries and further highlight salient regions. Experimental results over seven popular RGBD salient object detection benchmarks demonstrate that DGFSnet fairly locates salient regions and effectively segments the complete object.

A Machine Learning Framework for Assessing the Risk of Venous Thromboembolism in Patients Undergoing Hip or Knee Replacement.

Venous thromboembolism (VTE) is a well-recognized complication that is prevalent in patients undergoing major orthopedic surgery (e.g., total hip arthroplasty and total knee arthroplasty). For years, to identify patients at high risk of developing VTE, physicians have relied on traditional risk scoring systems, which are too simplistic to capture the risk level accurately. In this paper, we propose a data-driven machine learning framework to identify such high-risk patients before they undergo a major hip or knee surgery. Using electronic health records of more than 392,000 patients who undergone a major orthopedic surgery, and following a guided feature selection using the genetic algorithm, we trained a fully connected deep neural network model to predict high-risk patients for developing VTE. We identified several risk factors for VTE that were not previously recognized. The best FCDNN model trained using the selected features yielded an area under the ROC curve (AUC) of 0.873, which was remarkably higher than the best AUC obtained by including only risk factors previously known in the medical literature. Our findings suggest several interesting and important insights. The traditional risk scoring tables that are being widely used by physicians to identify high-risk patients are not considering a comprehensive set of risk factors, nor are they as powerful as cutting-edge machine learning methods in distinguishing low- from high-risk patients.

A 12-hospital prospective evaluation of a clinical decision support prognostic algorithm based on logistic regression as a form of machine learning to facilitate decision making for patients with suspected COVID-19.

To prospectively evaluate a logistic regression-based machine learning (ML) prognostic algorithm implemented in real-time as a clinical decision support (CDS) system for symptomatic persons under investigation (PUI) for Coronavirus disease 2019 (COVID-19) in the emergency department (ED). We developed in a 12-hospital system a model using training and validation followed by a real-time assessment. The LASSO guided feature selection included demographics, comorbidities, home medications, vital signs. We constructed a logistic regression-based ML algorithm to predict "severe" COVID-19, defined as patients requiring intensive care unit (ICU) admission, invasive mechanical ventilation, or died in or out-of-hospital. Training data included 1,469 adult patients who tested positive for Severe Acute Respiratory Syndrome Coronavirus 2 (SARS-CoV-2) within 14 days of acute care. We performed: 1) temporal validation in 414 SARS-CoV-2 positive patients, 2) validation in a PUI set of 13,271 patients with symptomatic SARS-CoV-2 test during an acute care visit, and 3) real-time validation in 2,174 ED patients with PUI test or positive SARS-CoV-2 result. Subgroup analysis was conducted across race and gender to ensure equity in performance. The algorithm performed well on pre-implementation validations for predicting COVID-19 severity: 1) the temporal validation had an area under the receiver operating characteristic (AUROC) of 0.87 (95%-CI: 0.83, 0.91); 2) validation in the PUI population had an AUROC of 0.82 (95%-CI: 0.81, 0.83). The ED CDS system performed well in real-time with an AUROC of 0.85 (95%-CI, 0.83, 0.87). Zero patients in the lowest quintile developed "severe" COVID-19. Patients in the highest quintile developed "severe" COVID-19 in 33.2% of cases. The models performed without significant differences between genders and among race/ethnicities (all p-values > 0.05). A logistic regression model-based ML-enabled CDS can be developed, validated, and implemented with high performance across multiple hospitals while being equitable and maintaining performance in real-time validation.

Segmentation of Cardiac Structures via Successive Subspace Learning with Saab Transform from Cine MRI.

Assessment of cardiovascular disease (CVD) with cine magnetic resonance imaging (MRI) has been used to non-invasively evaluate detailed cardiac structure and function. Accurate segmentation of cardiac structures from cine MRI is a crucial step for early diagnosis and prognosis of CVD, and has been greatly improved with convolutional neural networks (CNN). There, however, are a number of limitations identified in CNN models, such as limited interpretability and high complexity, thus limiting their use in clinical practice. In this work, to address the limitations, we propose a lightweight and interpretable machine learning model, successive subspace learning with the subspace approximation with adjusted bias (Saab) transform, for accurate and efficient segmentation from cine MRI. Specifically, our segmentation framework is comprised of the following steps: (1) sequential expansion of near-to-far neighborhood at different resolutions; (2) channel-wise subspace approximation using the Saab transform for unsupervised dimension reduction; (3) class-wise entropy guided feature selection for supervised dimension reduction; (4) concatenation of features and pixel-wise classification with gradient boost; and (5) conditional random field for post-processing. Experimental results on the ACDC 2017 segmentation database, showed that our framework performed better than state-of-the-art U-Net models with 200× fewer parameters in delineating the left ventricle, right ventricle, and myocardium, thus showing its potential to be used in clinical practice.Clinical relevance- Delineation of the left ventricular cavity, myocardium, and right ventricle from cardiac MR images is a common clinical task to establish diagnosis and prognosis of CVD.

A Role for Prior Knowledge in Statistical Classification of the Transition from Mild Cognitive Impairment to Alzheimer's Disease.

The transition from mild cognitive impairment (MCI) to dementia is of great interest to clinical research on Alzheimer's disease and related dementias. This phenomenon also serves as a valuable data source for quantitative methodological researchers developing new approaches for classification. However, the growth of machine learning (ML) approaches for classification may falsely lead many clinical researchers to underestimate the value of logistic regression (LR), which often demonstrates classification accuracy equivalent or superior to other ML methods. Further, when faced with many potential features that could be used for classifying the transition, clinical researchers are often unaware of the relative value of different approaches for variable selection. The present study sought to compare different methods for statistical classification and for automated and theoretically guided feature selection techniques in the context of predicting conversion from MCI to dementia. We used data from the Alzheimer's Disease Neuroimaging Initiative (ADNI) to evaluate different influences of automated feature preselection on LR and support vector machine (SVM) classification methods, in classifying conversion from MCI to dementia. The present findings demonstrate how similar performance can be achieved using user-guided, clinically informed pre-selection versus algorithmic feature selection techniques. These results show that although SVM and other ML techniques are capable of relatively accurate classification, similar or higher accuracy can often be achieved by LR, mitigating SVM's necessity or value for many clinical researchers.

A WGFS Based Approach to Extract Factors Influencing the Marketing of Korean Language in GCC

This research proposed an approach that is intended to determine the minimal set of important factors that influence the desire of learning Korean language in the Gulf Cooperation Council (GCC). Those factors will then influence marketing of the Korean language in GCC by guiding interested people to increase their commercial abilities, improve their information about Korean drama, and prepare them to study or travel to Korea. A total of 500 responses out of 526 questionnaires were used for the analysis process. Merging the weight by SVM and the weight guided feature selection (WGFS) techniques were proposed to build a strong hybrid model of reduction for the investigated dataset. Five different classifiers were used to test the results. Empirical results have showed that the generated factors (the reduct) are very significant to test the ability/inability of learning the Korean language. SVM was shown as the best with accuracy value of 94%. This research contributed to the literature by highlighting the importance of the Korean language in the GCC and by presenting the important factors that influence learners of the Korean language: encouragements and obstacles. Moreover, current research presented the best classifier which yields to the high performance of classification.

Identifying temporal molecular signatures underlying cardiovascular diseases: A data science platform

ObjectiveDuring cardiovascular disease progression, molecular systems of myocardium (e.g., a proteome) undergo diverse and distinct changes. Dynamic, temporally-regulated alterations of individual molecules underlie the collective response of the heart to pathological drivers and the ultimate development of pathogenesis. Advances in high-throughput omics technologies have enabled cost-effective, temporal profiling of targeted systems in animal models of human diseases. However, computational analysis of temporal patterns from omics data remains challenging. In particular, bioinformatic pipelines involving unsupervised statistical approaches to support cardiovascular investigations are lacking, which hinders one's ability to extract biomedical insights from these complex datasets. Approach and resultsWe developed a non-parametric data analysis platform to resolve computational challenges unique to temporal omics datasets. Our platform consists of three modules. Module I preprocesses the temporal data using either cubic splines or principal component analysis (PCA), and it simultaneously accomplishes the tasks on missing data imputation and denoising. Module II performs an unsupervised classification by K-means or hierarchical clustering. Module III evaluates and identifies biological entities (e.g., molecular events) that exhibit strong associations to specific temporal patterns. The jackstraw method for cluster membership has been applied to estimate p-values and posterior inclusion probabilities (PIPs), both of which guided feature selection. To demonstrate the utility of the analysis platform, we employed a temporal proteomics dataset that captured the proteome-wide dynamics of oxidative stress induced post-translational modifications (O-PTMs) in mouse hearts undergoing isoproterenol (ISO)-induced hypertrophy. ConclusionWe have created a platform, CV.Signature.TCP, to identify distinct temporal clusters in omics datasets. We presented a cardiovascular use case to demonstrate its utility in unveiling biological insights underlying O-PTM regulations in cardiac remodeling. This platform is implemented in an open source R package (https://github.com/UCLA-BD2K/CV.Signature.TCP).

Information gain directed genetic algorithm wrapper feature selection for credit rating

Financial credit scoring is one of the most crucial processes in the finance industry sector to be able to assess the credit-worthiness of individuals and enterprises. Various statistics-based machine learning techniques have been employed for this task. “Curse of Dimensionality” is still a significant challenge in machine learning techniques. Some research has been carried out on Feature Selection (FS) using genetic algorithm as wrapper to improve the performance of credit scoring models. However, the challenge lies in finding an overall best method in credit scoring problems and improving the time-consuming process of feature selection. In this study, the credit scoring problem is investigated through feature selection to improve classification performance. This work proposes a novel approach to feature selection in credit scoring applications, called as Information Gain Directed Feature Selection algorithm (IGDFS), which performs the ranking of features based on information gain, propagates the top m features through the GA wrapper (GAW) algorithm using three classical machine learning algorithms of KNN, Naïve Bayes and Support Vector Machine (SVM) for credit scoring. The first stage of information gain guided feature selection can help reduce the computing complexity of GA wrapper, and the information gain of features selected with the IGDFS can indicate their importance to decision making.Regarding the classification accuracy, SVM accuracy is always better than KNN and NB for Baseline techniques, GAW and IGDFS. Also, we can conclude that the IGDFS achieved better performance than generic GAW, and GAW obtained better performance than the corresponding single classifiers (baseline) for almost all cases, except for the German Credit dataset, IGDFS + KNN has worse performance than generic GAW and the single classifier KNN. Removing features with low information gain could produce conflict with the original data structure for KNN, and thus affect the performance of IGDFS + KNN.Regarding the ROC performance, for the German Credit Dataset, the three classic machine learning algorithms, SVM, KNN and Naïve Bayes in the wrapper of IGDFS GA obtained almost the same performance. For the Australian credit dataset and the Taiwan Credit dataset, the IGDFS + Naive Bayes achieved the largest area under ROC curves.

Eye tracking data guided feature selection for image classification

Feature selection has played a critical role in image classification, since it is able to remove irrelevant and redundant features and to eventually reduce the dimensionality of feature space. Although existing feature selection methods have achieved promising progress, human factors have seldom been taken into account. To tackle such a problem, a novel two-stage feature selection method is proposed for image classification by taking human factors into account and leveraging the value of eye tracking data. In the coarse selection stage, with the help of eye tracking data, Regions of Interests (ROIs) from the human perspective are first identified to represent an image with visual features. Then, with an improved quantum genetic algorithm (IQGA) that incorporates a novel mutation strategy for alleviating the premature convergence, a subset of features is obtained for the subsequent fine selection. In the fine selection stage, a hybrid method is proposed to integrate the efficiency of the minimal-Redundancy-Maximal-Relevance (mRMR) and the effectiveness of the Support Vector Machine based Recursive Feature Elimination (SVM-RFE). In particular, the ranking criterion of the SVM-RFE is improved by incorporating the ranking information obtained from the mRMR. Comprehensive experimental results in two benchmark datasets demonstrate that eye tracking data are of great importance to improve the performance of feature selection for image classification.

Temperature profile retrieval in axisymmetric combustion plumes using multilayer perceptron modeling and spectral feature selection in the infrared CO2 emission band.

In this work, a methodology based on the combined use of a multilayer perceptron model fed using selected spectral information is presented to invert the radiative transfer equation (RTE) and to recover the spatial temperature profile inside an axisymmetric flame. The spectral information is provided by the measurement of the infrared CO2 emission band in the 3-5 μm spectral region. A guided spectral feature selection was carried out using a joint criterion of principal component analysis and a priori physical knowledge of the radiative problem. After applying this guided feature selection, a subset of 17 wavenumbers was selected. The proposed methodology was applied over synthetic scenarios. Also, an experimental validation was carried out by measuring the spectral emission of the exhaust hot gas plume in a microjet engine with a Fourier transform-based spectroradiometer. Temperatures retrieved using the proposed methodology were compared with classical thermocouple measurements, showing a good agreement between them. Results obtained using the proposed methodology are very promising and can encourage the use of sensor systems based on the spectral measurement of the CO2 emission band in the 3-5 μm spectral window to monitor combustion processes in a nonintrusive way.

Constraint Score: A new filter method for feature selection with pairwise constraints

Feature selection is an important preprocessing step in mining high-dimensional data. Generally, supervised feature selection methods with supervision information are superior to unsupervised ones without supervision information. In the literature, nearly all existing supervised feature selection methods use class labels as supervision information. In this paper, we propose to use another form of supervision information for feature selection, i.e. pairwise constraints, which specifies whether a pair of data samples belong to the same class ( must-link constraints) or different classes ( cannot-link constraints). Pairwise constraints arise naturally in many tasks and are more practical and inexpensive than class labels. This topic has not yet been addressed in feature selection research. We call our pairwise constraints guided feature selection algorithm as Constraint Score and compare it with the well-known Fisher Score and Laplacian Score algorithms. Experiments are carried out on several high-dimensional UCI and face data sets. Experimental results show that, with very few pairwise constraints, Constraint Score achieves similar or even higher performance than Fisher Score with full class labels on the whole training data, and significantly outperforms Laplacian Score.

Fast leave-one-out evaluation for dynamic gene selection

Gene selection procedure is a necessary step to increase the accuracy of machine learning algorithms that help in disease diagnosis based on gene expression data. This is commonly known as a feature subset selection problem in machine learning domain. A fast leave-one-out (LOO) evaluation formula for least-squares support vector machines (LS-SVMs) is introduced here that can guide our backward feature selection process. Based on that, we propose a fast LOO guided feature selection (LGFS) algorithm. The gene selection step size is dynamically adjusted according to the LOO accuracy estimation. For our experiments, the application of LGFS to the gene selection process improves the classifier accuracy and reduces the number of features required as well. The least number of genes that can maximize the disease classification accuracy is automatically determined by our algorithm.