Supervised Feature Selection Algorithm Research Articles

Mechanism of rainfall induced landslides in Yunnan Province using multi-scale spatiotemporal analysis and remote sensing interpretation

In order to explore the disaster mechanism of rainfall induced landslides in Yunnan Province, a new method is developed in the field of landslide research, which provides a strong basis for the study of rainfall induced landslide disaster mechanism in different regions. First, the mechanism of rainfall induced landslide initiation and the mechanism of rainfall impacting slope are introduced, and the temporal and spatial characteristics of landslide disasters in Yunnan Province are analyzed comprehensively. The influence of rainfall on landslide density is analyzed in 8 regions randomly, and a semi supervised feature selection algorithm based on manifold is proposed to optimize the remote sensing image of Zhaojiapo rainfall type landslide area in Yunnan Province. The results show that in the province the monthly rainfall days account for a large proportion from June to September, and the months with the largest number of landslides in the whole year are from June to August. The seasonal change of rainfall belt and the frequent occurrence time of landslide disasters and the spatial characteristics of landslide distribution and rainfall distribution basically belong to corresponding relationship; the greater the rainfall of 8 regions, the greater the probability of landslide occurrence, and the greater the landslide density, which belongs to the proportional relationship; the landform characteristics in the optimized remote sensing image can be clearly displayed, and the classification accuracy of the data is enhanced. The results show that based on multi-scale spatiotemporal analysis combined with optimized remote sensing technology, the cause of rainfall induced landslides in Yunnan Province is the change of landslide density caused by heavy rainfall, which eventually leads to disasters. It is conducive to scientifically understanding the inducement of rainfall induced landslides in Yunnan Province, provides a scientific basis for further disaster control and prevention, and provides a new possible method for landslide real-time prediction and alarm system.

Deep learning assisted multi-omics integration for survival and drug-response prediction in breast cancer

BackgroundSurvival and drug response are two highly emphasized clinical outcomes in cancer research that directs the prognosis of a cancer patient. Here, we have proposed a late multi omics integrative framework that robustly quantifies survival and drug response for breast cancer patients with a focus on the relative predictive ability of available omics datatypes. Neighborhood component analysis (NCA), a supervised feature selection algorithm selected relevant features from multi-omics datasets retrieved from The Cancer Genome Atlas (TCGA) and Genomics of Drug Sensitivity in Cancer (GDSC) databases. A Neural network framework, fed with NCA selected features, was used to develop survival and drug response prediction models for breast cancer patients. The drug response framework used regression and unsupervised clustering (K-means) to segregate samples into responders and non-responders based on their predicted IC50 values (Z-score).ResultsThe survival prediction framework was highly effective in categorizing patients into risk subtypes with an accuracy of 94%. Compared to single-omics and early integration approaches, our drug response prediction models performed significantly better and were able to predict IC50 values (Z-score) with a mean square error (MSE) of 1.154 and an overall regression value of 0.92, showing a linear relationship between predicted and actual IC50 values.ConclusionThe proposed omics integration strategy provides an effective way of extracting critical information from diverse omics data types enabling estimation of prognostic indicators. Such integrative models with high predictive power would have a significant impact and utility in precision oncology.

Machine Learning Based Stress Monitoring in Older Adults Using Wearable Sensors and Cortisol as Stress Biomarker

The objective of this work is to evaluate the effectiveness of a wearable physiological stress monitoring system in distinguishing between stressed and non-stressed state in older adults using machine learning techniques. This system utilizes EDA and BVP signal to detect occurrence of stress as indicated by salivary cortisol measurement which is a reliable objective measure of physiological stress. Data of 19 healthy older adults (11 female and 8 male) with mean age 73.15 ± 5.79 were used for this study. EDA and BVP signals were recorded using a finger tip sensor during the Trier Social Stress Test, which is a well known experimental protocol to reliably induce stress in humans in a social setting. 39 statistical measures of the peak characteristic of EDA and BVP signal were extracted. A supervised feature selection algorithm is used to select important features as an input to the machine learning model. Four machine learning algorithms were evaluated based on their performance in classifying between stressed and non-stressed states. Results indicate that the logistic regression performed the best among Random Forest, κ-NN, and Support Vector Machine achieving an macro-average and micro-average f1-score of 0.87 and 0.95 respectively and an AUC score of 0.81. We also evaluated the effectiveness of a novel deep learning Long Short-Term Memory (LSTM) based classifier in distinguishing between stressed and non-stressed state. Results on test data shows that LSTM based classifier achieved an improvement of 6.7% and 2% in terms of macro-average f1-score and micro-average f1-score respectively. Also the AUC score for LSTM classifier is found to be 0.9 which is about 11% higher than the best performing logistic regression model. This work can be used to design a convenient unobtrusive wearable device to monitor stress levels in older adults in their home environment, thereby facilitating aging in place and improving the quality of life.

Stable feature selection using copula based mutual information

Feature selection is a key step in many machine learning tasks. A majority of the existing methods of feature selection address the problem by devising some scoring function while treating the features independently, thereby overlooking their interdependencies. We leverage the scale invariance property of copula to construct a greedy, supervised feature selection algorithm that maximizes the feature relevance while minimizing the redundant information content. Multivariate copula is used in the proposed copula Based Feature Selection (CBFS) to discover the dependence structure between features. The incorporation of copula-based multivariate dependency in the formulation of mutual information helps avoid averaging over multiple instances of bivariate dependencies, thus eliminating the average estimation error introduced when bivariate dependency is used between a pair of feature variables. Under a controlled setting, our algorithm outperformed the existing best practice methods in warding off the noise in data. On several real and synthetic datasets, the proposed algorithm performed competitively in maximizing classification accuracy. CBFS also outperforms the other methods in terms of its noise tolerance property.

Machine Learning Based Supervised Feature Selection Algorithm for Data Mining

Data Scientists focus on high dimensional data to predict and reveal some interesting patterns as well as most useful information to the modern world. Feature Selection is a preprocessing technique which improves the accuracy and efficiency of mining algorithms. There exist a numerous feature selection algorithms. Most of the algorithms failed to give better mining results as the scale increases. In this paper, feature selection for supervised algorithms in data mining are considered and given an overview of existing machine learning algorithm for supervised feature selection. This paper introduces an enhanced supervised feature selection algorithm which selects the best feature subset by eliminating irrelevant features using distance correlation and redundant features using symmetric uncertainty. The experimental results show that the proposed algorithm provides better classification accuracy and selects minimum number of features.

Supervised feature selection via matrix factorization based on singular value decomposition

Feature selection is a main research issue in gene expression data. It tries to determine the best subset of features (genes) for a machine learning model or statistical analysis. In this paper, a novel supervised feature selection algorithm is proposed by using matrix factorization and Singular Value Decomposition (SVD) for microarray datasets. First, an unsupervised matrix factorization based existed method in the literature is used for feature selection as the main framework of our proposed method. Then SVD is incorporated into the first matrix factorization approach for increasing its efficiency. In SVD, a matrix decomposes into three matrices such that one of these factors contains an orthonormal basis for original matrix. Thus, a submatrix of the original matrix is located instead of this factor. The replaced submatrix produces the same space as original matrix, therefore without any complex measure we find a submatrix that has the least distance with data matrix. By considering linear independence as a redundancy criterion, selected features have the least redundancy between themselves. To improve prediction accuracy, the sum of Information Gain (IG) values for selected features is formulated in the objective function as the relevancy measure and makes our algorithm to be a supervised approach. Finally, the proposed algorithm is compared with eleven common feature selection methods and two state-of-the-art approaches using nine publicly available microarray datasets. In terms of accuracy, the experimental results indicate that the proposed method is comparable with the state-of-the-art methods and it has the better performance than the others.

Supervised feature selection algorithm via discriminative ridge regression

This paper studies a new feature selection method for data classification that efficiently combines the discriminative capability of features with the ridge regression model. It first sets up the global structure of training data with the linear discriminant analysis that assists in identifying the discriminative features. And then, the ridge regression model is employed to assess the feature representation and the discrimination information, so as to obtain the representative coefficient matrix. The importance of features can be calculated with this representative coefficient matrix. Finally, the new subset of selected features is applied to a linear Support Vector Machine for data classification. To validate the efficiency, sets of experiments are conducted with twenty benchmark datasets. The experimental results show that the proposed approach performs much better than the state-of-the-art feature selection algorithms in terms of the evaluating indicator of classification. And the proposed feature selection algorithm possesses a competitive performance compared with existing feature selection algorithms with regard to the computational cost.

Hypergraph expressing low-rank feature selection algorithm

Dimensionality reduction has been attracted extensive attention in machine learning. It usually includes two types: feature selection and subspace learning. Previously, many researchers have demonstrated that the dimensionality reduction is meaningful for real applications. Unfortunately, a large mass of these works utilize the feature selection and subspace learning independently. This paper explores a novel supervised feature selection algorithm by considering the subspace learning. Specifically, this paper employs an l 2,1−norm and an l 2,p −norm regularizers, respectively, to conduct sample denoising and feature selection via exploring the correlation structure of data. Then this paper uses two constraints (i.e. hypergraph and low-rank) to consider the local structure and the global structure among the data, respectively. Finally, this paper uses the optimizing framework to iteratively optimize each parameter while fixing the other parameter until the algorithm converges. A lot of experiments show that our new supervised feature selection method can get great results on the eighteen public data sets.

Adaptive structure learning for low-rank supervised feature selection

Previous Spectral Feature Selection (SFS) methods output promising feature selection results in many real-world applications, which deeply depend on the preservation of the local or global structures of the data via learning a graph matrix. However, current SFS methods 1) learn the graph matrix in the original data which may contain a number of noise to affect the results of feature selection, 2) conduct the learning of a low-dimensional feature space and the graph matrix individually, thus hard achieve the optimal results of feature selection even though both of these two steps achieve their individual optimization, and 3) consider either the local or global structure of data to difficult provide complementary information for feature selection. To address the above issues, this paper proposes a novel supervised feature selection algorithm to simultaneously preserve the local structure (via adaptive structure learning in a low-dimensional feature space of the original data) and the global structure (via a low-rank constraint) of the data. Moreover, we also propose a new optimization method to fast optimize the resulting objective function. We finally verify the proposed method on eight real-word and benchmark datasets, by comparing with the state-of-the-art feature selection methods, and experimental results show that our proposed method achieves competitive results in term of classification performance.

GMDH-based semi-supervised feature selection for customer classification

Data dimension reduction is an important step for customer classification modeling, and feature selection has been a research focus of the data dimension reduction field. This study introduces the group method of data handling (GMDH), puts forward a GMDH-based semi-supervised feature selection (GMDH-SSFS) algorithm, and applies it to customer feature selection. The algorithm can utilize a few samples with class labels L, and a large number of samples without class labels U simultaneously. What is more, it considers the relationship between features and class labels, and that between features during feature selection. The GMDH-SSFS model mainly consists of three stages: 1) Train N basic classification models based on the dataset L with class labels; 2) Label samples selectively in the dataset U without class labels, and add them to L; 3) Train the GMDH neural network based on the new training set L, and select the optimal feature subset FS. Based on an empirical analysis of four customer classification datasets, results suggest that the features selected by the GMDH-SSFS model have a good explainability. Meanwhile, the customer classification performance of the classification model trained by the selected feature subset is superior to that of the models trained by the commonly used Laplacian score (an unsupervised feature selection algorithm), Fisher score (a supervised feature selection algorithm), and the FW-SemiFS and S3VM-FS (two semi-supervised feature selection algorithms).

Internet Traffic Classification Using Machine Learning

Internet traffic classification is one of the popular research interest area because of its benefits for many applications like intrusion detection system, congestion avoidance, traffic prediction etc. Internet traffic is classified on the basis of statistical features because port and payload based techniques have their limitations. For statistics based techniques machine learning is used. The statistical feature set is large. Hence, it is a challenge to reduce the large feature set to an optimal feature set. This will reduce the time complexity of the machine learning algorithm. This paper tries to obtain an optimal feature set by using a hybrid approach -An unsupervised clustering algorithm (K-Means) with a supervised feature selection algorithm (Best Feature Selection).

Hybrid Tolerance Rough Set–Firefly based supervised feature selection for MRI brain tumor image classification

Brain tumor is one of the most harmful diseases, and has affected majority of people including children in the world. The probability of survival can be enhanced if the tumor is detected at its premature stage. The intention of feature selection approach is to select a small subset of features which minimizes redundancy and maximizes relevance to the target such as the class labels in classification. Thus, the machine learning model receives a brief organization with high predictive accuracy using the selected prominent features. Therefore, currently, feature selection plays a significant role in machine learning and knowledge discovery. A novel hybrid supervised feature selection algorithm, called TRSFFQR (Tolerance Rough Set Firefly based Quick Reduct), is developed and applied for MRI brain images. The hybrid intelligent system aims to exploit the benefits of the basic models and at the same time, moderate their limitations. Different categories of features are extracted from the segmented MRI images, i.e., shape, intensity and texture based features. The features extracted from brain tumor Images are real values. Hence Tolerance Rough set is applied in this work. In this study, a hybridization of two techniques, Tolerance Rough Set (TRS) and Firefly Algorithm (FA) are used to select the imperative features of brain tumor. Performance of TRSFFQR is compared with Artificial Bee Colony (ABC), Cuckoo Search Algorithm (CSA), Supervised Tolerance Rough Set–PSO based Relative Reduct (STRSPSO-RR) and Supervised Tolerance Rough Set–PSO based Quick Reduct (STRSPSO-QR).The experimental result shows the effectiveness of the proposed technique as well as improvements over the existing supervised feature selection algorithms.

Feature selection with SVD entropy: Some modification and extension

Many approaches have been developed for dimensionality reduction. These approaches can broadly be categorized into supervised and unsupervised methods. In case of supervised dimensionality reduction, for any input vector the target value is known, which can be a class label also. In a supervised approach, our objective is to select a subset of features that has adequate discriminating power to predict the target value. This target value for an input vector is absent in case of an unsupervised approach. In an unsupervised scheme, we mainly try to find a subset that can capture the inherent “structure” of the data, such as the neighborhood relation or the cluster structure. In this work, we first study a Singular Value Decomposition (SVD) based unsupervised feature selection approach proposed by Varshavsky et al. Then we propose a modification of this method to improve its performance. An SVD-entropy based supervised feature selection algorithm is also developed in this paper. Performance evaluation of the algorithms is done on altogether 13 benchmark and one Synthetic data sets. The quality of the selected features is assessed using three indices: Sammon’s Error (SE), Cluster Preservation Index (CPI) and MisClassification Error (MCE) using a 1-Nearest Neighbor (1-NN) classifier. Besides showing the improvement of the modified unsupervised scheme over the existing one, we have also made a comparative study of the modified unsupervised and the proposed supervised algorithms with one well-known unsupervised and two popular supervised feature selection methods respectively. Our results reveal the effectiveness of the proposed algorithms in selecting relevant features.