Effectiveness Of Feature Selection Method Research Articles

Evaluation of the Effectiveness of Feature Selection Methods Combined with Regression Algorithms to Predict Particulate Matter (PM10) in Gandhinagar, Gujarat, India

Feature selection is one of the important data pre-processing techniques that are used to increase the performance of machine learning models, to build faster and more cost-effective algorithms, and to make it easier to interpret the predictions made by the models. The main objective of this research work is to investigate the influence features to predict particulate matter (PM10). This research uses 24-hour average pollutant concentration data of 36 air quality monitoring stations provided by Gandhinagar Smart City Development Limited (GSCDL), Gandhinagar, Gujarat. Important features were identified using five feature selection techniques (correlation, forward selection, backward elimination, Exhaustive Feature Selection (EFS), and feature importance derived using Random Forest Regressor). With selected features six regression algorithms (Multiple Linear Regression, Random Forest, Decision Tree, K-nearest Neighbour, XGBoost, and Support Vector Regressor) were trained to predict PM10. Further, the models were compared based on the Root Mean Square Error (RMSE) and Coefficient of determination (R2) parameters to identify the model with good performance. This proposed model can be utilized as an early warning system, providing air quality information to local authorities to develop air-quality improvement initiatives.

The Effectiveness of Feature Selection Methods for Imbalanced Text Classification

Metin verilerinin sınıflar arasında dağılımı genellikle eşit değildir. Bu durum, metin sınıflandırma 
 işleminde sınıflandırıcıların performansına olumsuz yansımaktadır. Dengesiz metin sınıflandırma ile ilgili 
 birçok çalışma yapılmıştır. Metin sınıflandırma işleminin önemli aşamalarından olan öznitelik seçim 
 aşaması, dengesiz metin probleminde de kritik öneme sahiptir. Öznitelik seçme metotlarının dengesiz 
 metinlerin sınıflandırılması üzerindeki etkisi bu çalışmada etraflıca araştırılmıştır. Bu doğrultuda, iki 
 farklı veri seti üzerinde üç farklı sınıflandırıcı ve dokuz farklı öznitelik seçim metodu ile birçok deney 
 yapılmıştır. Ayrıca öznitelik seçim yöntemlerinin başarıları farklı öznitelik sayılarında da gözlemlenmiştir. 
 NDM, DFSS, PFS, POISSON, CHI2, IG, GINI, DFS ve MDFS olarak adlandırılan 9 farklı öznitelik seçim 
 metodu değerlendirilmiştir. Destek Vektör Makinesi (SVM), Karar Ağacı (DTREE) ve Basit Bayes (MNB) 
 sınıflandırıcıları ile deneysel sonuçlar elde edilmiştir. Reuters-21578 veri setinde DFS ve CHI2 öznitelik 
 seçim yöntemleri Makro-F1 değerlendirme metriği üzerinden yaklaşık en yüksek 80 değerini alırken, 
 SPAM SMS veri setinde, DFS öznitelik seçim yöntemi en yüksek skor olarak 95 ve CHI2 öznitelik seçim 
 yöntemi 94 değerlerini almıştır. Öznitelik seçme metotlarından DFS ve CHI2’nin dengesiz metin 
 sınıflandırmada daha başarılı olduğu görülmektedir.

Cyberattacks Detection Through Behavior Analysis of Internet Traffic

Network intrusion detection systems (NIDS) are actually used to detect suspicious activities such as viruses, shellcode, XSS, CSRF, worms, etc. There are two types of the NIDS: signature-based and anomaly-based. Recently, Deep Learning have emerged as promising techniques for classifying network attacks. In this paper, we propose a method to analyze the network traffic behavior through Deep Learning classification techniques using traffic features. The results indicate that Multi-Layer Perceptron (MLP) and Convolutional Neural Network (CNN) techniques achieved similar performance with 94% accuracy when using all features in the used dataset. However, with the use of feature selection techniques such as XGBoost, Pearson correlation, and mutual information, the models achieved a slightly lower accuracy of 91%, but these results demonstrate the effectiveness of feature selection methods in enhancing the performance of Deep Learning models by reducing complexity and removing irrelevant features.

Predicting Emotional Valence of People Living with the Human Immunodeficiency Virus Using Daily Voice Clips: A Preliminary Study.

To detect depression in people living with the human immunodeficiency virus (PLHIV), this preliminary study developed an artificial intelligence (AI) model aimed at discriminating the emotional valence of PLHIV. Sixteen PLHIV recruited from the Taoyuan General Hospital, Ministry of Health and Welfare, participated in this study from 2019 to 2020. A self-developed mobile application (app) was installed on sixteen participants’ mobile phones and recorded their daily voice clips and emotional valence values. After data preprocessing of the collected voice clips was conducted, an open-source software, openSMILE, was applied to extract 384 voice features. These features were then tested with statistical methods to screen critical modeling features. Several decision-tree models were built based on various data combinations to test the effectiveness of feature selection methods. The developed model performed very well for individuals who reported an adequate amount of data with widely distributed valence values. The effectiveness of feature selection methods, limitations of collected data, and future research were discussed.

Impact of Feature Selection for Data Classification Using Naive Bayes Classifier

In the field of data processing and analysis, the dataset may be a large set of features that restrict data usability and applicability, and thus the dimensions of data sets need to be reduced. Feature selection is the process of removing as much of the redundant and irrelevant features as possible from the original dataset to improve the mining process efficiency. This paper presented a study to evaluate and compare the effect of filter and wrapper methods as feature selection approaches in terms of classification accuracy and time complexity. The Naive Bayes Classifier and three classification datasets from the UCI repository are utilizing in the classification procedure. To investigate the effect of feature selection methods, they are applied to the different characteristics datasets to obtain the selected feature vectors which are then classified according to each dataset category. The datasets used in this paper are the Iris, Ionosphere, and Ovarian Cancer dataset. Experimental results indicate that the filter and wrapper methods provide approximately equal classification accuracy where the average accuracy value of the Ionosphere and Ovarian Cancer dataset is 0.78 and 0.91 for the same selected feature vectors respectively. For Iris dataset, the filter method outperforms the wrapper method by achieving the same accuracy value using only half number of selected features. The results also show that the filter method surpasses when considering the execution time.

A novel hybrid approach based on relief algorithm and fuzzy reinforcement learning approach for predicting wind speed

Wind speed (WS) prediction has become popular nowadays due to increasing demand for wind power generation and competitive development in wind energy. Many prediction models are used to predict WS for which wind is non-stationary, nonlinear and irregular. However, they neglect the effectiveness of feature selection methods in WS prediction, thereby creating very challenging for precise prediction of WS and safe operation of the wind industry. To overpower these challenges and further improve WS prediction accuracy, a prediction model is developed based on feature selection technique and prediction models. Therefore this study proposes an adaptive self-learning wind speed (WS) predicting model using fuzzy reinforcement learning (FRL) that is Fuzzy Q Learning (FQL). Proposed FQL based WS predictor model can predict with great accuracy. This is a first effort at developing a forecasting model using FRL for WS prediction. The presented model has no prior knowledge of the system or plant or target speed information. Measured WS is processed through Info Gain attribute evaluator with Ranker search method feature selection purpose which serves as input to the FQL based WS prediction model. The comparison of proposed prediction method and existing machine learning based is carried out using simulations. The performance analysis indicates that the proposed method serves as an important tool for wind potential assessment.

RETRACTED ARTICLE: Automatic classification of emotions in news articles through ensemble decision tree classification techniques

Emotions form a major role in human life. As human interactions with online systems have increased drastically, emotion prediction from online text, which otherwise can be monotonous, would help to provide a better environment to the users. Identification of emotions from a normal text itself is very complicated while news text that does not explicitly convey emotions adds more intricacy to it. Data mining methods can be utilized in this context. In this work, the potential of decision tree classifiers in emotion classification is explored. The advocated methodology incorporates two segments towards emotion identification. The first segment deals with data preparation and involves dataset elicitation, translation, HTML tag removal, stop word elimination and stemming. The second segment that implements data mining takes the output of the first segment as its input and applies feature vector formulation, correlation based feature selection, building of bagged Grafted C4.5 learning model and performance evaluation. Based on the evolved classification rules, the emotions are categorized into joy, surprise, fear, sadness, disgust, neutral and mixed kind. Experiments have been conducted to analyse the effect of feature selection methods and ensemble methods in generating efficient rules. The accuracy is compared against eight other decision tree classifiers and also the support vector machine learning model. The proposed methodology achieves the maximum accuracy of 87.83% justifying its utilization in the real time applications.

Impact of Feature Selection Methods on the Predictive Performance of Software Defect Prediction Models: An Extensive Empirical Study

Feature selection (FS) is a feasible solution for mitigating high dimensionality problem, and many FS methods have been proposed in the context of software defect prediction (SDP). Moreover, many empirical studies on the impact and effectiveness of FS methods on SDP models often lead to contradictory experimental results and inconsistent findings. These contradictions can be attributed to relative study limitations such as small datasets, limited FS search methods, and unsuitable prediction models in the respective scope of studies. It is hence critical to conduct an extensive empirical study to address these contradictions to guide researchers and buttress the scientific tenacity of experimental conclusions. In this study, we investigated the impact of 46 FS methods using Naïve Bayes and Decision Tree classifiers over 25 software defect datasets from 4 software repositories (NASA, PROMISE, ReLink, and AEEEM). The ensuing prediction models were evaluated based on accuracy and AUC values. Scott–KnottESD and the novel Double Scott–KnottESD rank statistical methods were used for statistical ranking of the studied FS methods. The experimental results showed that there is no one best FS method as their respective performances depends on the choice of classifiers, performance evaluation metrics, and dataset. However, we recommend the use of statistical-based, probability-based, and classifier-based filter feature ranking (FFR) methods, respectively, in SDP. For filter subset selection (FSS) methods, correlation-based feature selection (CFS) with metaheuristic search methods is recommended. For wrapper feature selection (WFS) methods, the IWSS-based WFS method is recommended as it outperforms the conventional SFS and LHS-based WFS methods.

A Novel Feature Selection Method for the Dynamic Security Assessment of Power Systems Based on Multi-Layer Perceptrons

In this study, the effect of feature selection methods on the performance of multi-layer perceptrons used for the dynamic security assessment of electric power systems is investigated. The existence of many measurable parameters (features) characterizing the power system security status complicates the use of multi-layer perceptron both in terms of prediction accuracy and training time. In this paper, the dynamic security of a power system subject to a number of critical contingencies is assessed as the critical clearing time of any credible fault is predicted by a multi-layer perceptron. In addition to the study of two different feature selection methods, which are Minimum Redundancy Maximum Relevance (mRMR), and Regressional ReliefF (RReliefF), a novel multi-layer perceptron based feature selection method is proposed to be applied in the prediction of security indices. The performance of the feature selection methods on the dynamic security assessment is investigated on a 16-generator, 68-bus test system.

Resampling Imbalanced Class and the Effectiveness of Feature Selection Methods for Heart Failure Dataset

The real dataset has many shortcomings that pose challenges to machine learning High dimensional and imbalanced class prevalence is two important challenges Hence the classification of data is negatively impacted by imbalanced data and high dimensional could create suboptimal performance of the classifier In this paper we explore and analyse different feature selection methods for a clinical dataset that suffers from high dimensional and imbalance data The aim of this paper is to investigate the effect of imbalanced data on selecting features by implementing the feature selection methods to select a subset of the original data and then resample the dataset In addition we resample the dataset to apply feature selection methods on a balanced class to compare the results with the original data Random forest and J techniques were used to evaluate the efficacy of samples The experiments confirm that resampling imbalanced class obtains a significant increase in classification performance for both taxonomy methods Random forest and J Furthermore the biggest measure affected by balanced data is specificity where it is sharply increased for all methods What is more the subsets selected from the balanced data just improve the performance for information gain where it is played down for the performance of others nbsp

A comparison of feature selection methods when using motion sensors data: a case study in Parkinson's disease.

The objective of this study is to investigate the effects of feature selection methods on the performance of machine learning methods for quantifying motor symptoms of Parkinson's disease (PD) patients. Different feature selection methods including step-wise regression, Lasso regression and Principal Component Analysis (PCA) were applied on 88 spatiotemporal features that were extracted from motion sensors during hand rotation tests. The selected features were then used in support vector machines (SVM), decision trees (DT), linear regression, and random forests models to calculate a so-called treatment-response index (TRIS). The validity, testretest reliability and sensitivity to treatment were assessed for each combination (feature selection method plus machine learning method). There were improvements in correlation coefficients and root mean squared error (RMSE) for all the machine learning methods, except DTs, when using the selected features from step-wise regression inputs. Using step-wise regression and SVM was found to have better sensitivity to treatment and higher correlation to clinical ratings on the Unified PD Rating Scale as compared to the combination of PCA and SVM. When assessing the ability of the machine learning methods to discriminate between tests performed by PD patients and healthy controls the results were mixed. These results suggest that the choice of feature selection methods is crucial when working with data-driven modelling. Based on our findings the step-wise regression can be considered as the method with the best performance.

On the effectiveness of feature selection methods for gait classification under different covariate factors

Gait classification is the problem of recognising individuals by the way in which they walk. The presence of covariate factors such as different clothing types, carrying conditions, walking surfaces, etc., can seriously complicate the task. Clothing, for instance, can occlude a significant amount of gait features and make human recognition difficult. Since the location of occlusion may differ for different covariate factors, relevant gait features may become irrelevant when the covariate factor changes, and exploiting occluded gait features can hinder the recognition performance. Therefore, feature selection has become an important step to make the analysis more manageable and to extract useful information for the gait classification task. Nevertheless, although feature selection is often used in order to identify the relevant body parts, to the best of our knowledge, a comparative analysis of feature selection techniques in gait recognition is seldom addressed. In this paper, we present an empirical approach to evaluate the degree of consistency among the performance of different selection algorithms in the context of gait identification under the effect of various covariate factors. First, a model-based framework for extracting informative gait features is introduced, then, an extensive comparative analysis of feature selection approaches in gait recognition is carried out. We perform a statistical study via ANOVA and mixed-effects models to examine the effect of six popular selection feature methods across classifiers and covariates. In addition, we systematically compare the selected feature subsets and the computational cost of the different selection approaches. The implemented method addresses the problem of feature selection for gait recognition on two well-known benchmark databases: the SOTON covariate database and the CASIA-B dataset, respectively. The investigated approach is able to select the discriminative input gait features and achieve an improved classification accuracy on par with other state-of-the-art methods.

Quantitative structure–activity relationship of the curcumin-related compounds using various regression methods

Quantitative structure activity relationship were used to study a series of curcumin-related compounds with inhibitory effect on prostate cancer PC-3 cells, pancreas cancer Panc-1 cells, and colon cancer HT-29 cells. Sphere exclusion method was used to split data set in two categories of train and test set. Multiple linear regression, principal component regression and partial least squares were used as the regression methods. In other hand, to investigate the effect of feature selection methods, stepwise, Genetic algorithm, and simulated annealing were used. In two cases (PC-3 cells and Panc-1 cells), the best models were generated by a combination of multiple linear regression and stepwise (PC-3 cells: r2 = 0.86, q2 = 0.82, pred_r2 = 0.93, and r2m (test) = 0.43, Panc-1 cells: r2 = 0.85, q2 = 0.80, pred_r2 = 0.71, and r2m (test) = 0.68). For the HT-29 cells, principal component regression with stepwise (r2 = 0.69, q2 = 0.62, pred_r2 = 0.54, and r2m (test) = 0.41) is the best method. The QSAR study reveals descriptors which have crucial role in the inhibitory property of curcumin-like compounds. 6ChainCount, T_C_C_1, and T_O_O_7 are the most important descriptors that have the greatest effect. With a specific end goal to design and optimization of novel efficient curcumin-related compounds it is useful to introduce heteroatoms such as nitrogen, oxygen, and sulfur atoms in the chemical structure (reduce the contribution of T_C_C_1 descriptor) and increase the contribution of 6ChainCount and T_O_O_7 descriptors. Models can be useful in the better design of some novel curcumin-related compounds that can be used in the treatment of prostate, pancreas, and colon cancers.

An Efficient Feature Selection Method for Arabic Text Classification

This paper proposes an efficient, Chi-Square-based, feature selection method for Arabic text classification. In Data Mining, feature selection is a preprocessing step that can improve the classification performance. Although few works have studied the effect of feature selection methods on Arabic text classification, limited number of methods was compared. Furthermore, different datasets were used by different works. This paper improves the previous works in three aspects. First, it proposes a new efficient feature selection method for enhancing Arabic text classification. Second, it compares extended number of existing feature selection methods. Third, it adopts two publicly available datasets to encourage future works to adopt them in order to guarantee fair comparisons among the various works. Our experiments show that our proposed method outperformed the existing methods in term of accuracy.

The Effectiveness of Feature Selection Method in Solar Power Prediction

This paper empirically shows that the effect of applying selected feature subsets on machine learning techniques significantly improves the accuracy for solar power prediction. Experiments are performed using five well-known wrapper feature selection methods to obtain the solar power prediction accuracy of machine learning techniques with selected feature subsets. For all the experiments, the machine learning techniques, namely, least median square (LMS), multilayer perceptron (MLP), and support vector machine (SVM), are used. Afterwards, these results are compared with the solar power prediction accuracy of those same machine leaning techniques (i.e., LMS, MLP, and SVM) but without applying feature selection methods (WAFS). Experiments are carried out using reliable and real life historical meteorological data. The comparison between the results clearly shows that LMS, MLP, and SVM provide better prediction accuracy (i.e., reduced MAE and MASE) with selected feature subsets than without selected feature subsets. Experimental results of this paper facilitate to make a concrete verdict that providing more attention and effort towards the feature subset selection aspect (e.g., selected feature subsets on prediction accuracy which is investigated in this paper) can significantly contribute to improve the accuracy of solar power prediction.

Phenotype prediction from genome-wide association studies: application to smoking behaviors

BackgroundA great success of the genome wide association study enabled us to give more attention on the personal genome and clinical application such as diagnosis and disease risk prediction. However, previous prediction studies using known disease associated loci have not been successful (Area Under Curve 0.55 ~ 0.68 for type 2 diabetes and coronary heart disease). There are several reasons for poor predictability such as small number of known disease-associated loci, simple analysis not considering complexity in phenotype, and a limited number of features used for prediction.MethodsIn this research, we investigated the effect of feature selection and prediction algorithm on the performance of prediction method thoroughly. In particular, we considered the following feature selection and prediction methods: regression analysis, regularized regression analysis, linear discriminant analysis, non-linear support vector machine, and random forest. For these methods, we studied the effects of feature selection and the number of features on prediction. Our investigation was based on the analysis of 8,842 Korean individuals genotyped by Affymetrix SNP array 5.0, for predicting smoking behaviors.ResultsTo observe the effect of feature selection methods on prediction performance, selected features were used for prediction and area under the curve score was measured. For feature selection, the performances of support vector machine (SVM) and elastic-net (EN) showed better results than those of linear discriminant analysis (LDA), random forest (RF) and simple logistic regression (LR) methods. For prediction, SVM showed the best performance based on area under the curve score. With less than 100 SNPs, EN was the best prediction method while SVM was the best if over 400 SNPs were used for the prediction.ConclusionsBased on combination of feature selection and prediction methods, SVM showed the best performance in feature selection and prediction.

The effect of feature selection methods on computer-aided detection of masses in mammograms

In computer-aided diagnosis (CAD) research, feature selection methods are often used to improve generalization performance of classifiers and shorten computation times. In an application that detects malignant masses in mammograms, we investigated the effect of using a selection criterion that is similar to the final performance measure we are optimizing, namely the mean sensitivity of the system in a predefined range of the free-response receiver operating characteristics (FROC). To obtain the generalization performance of the selected feature subsets, a cross validation procedure was performed on a dataset containing 351 abnormal and 7879 normal regions, each region providing a set of 71 mass features. The same number of noise features, not containing any information, were added to investigate the ability of the feature selection algorithms to distinguish between useful and non-useful features. It was found that significantly higher performances were obtained using feature sets selected by the general test statistic Wilks' lambda than using feature sets selected by the more specific FROC measure. Feature selection leads to better performance when compared to a system in which all features were used.

Improve feature selection method of web page language identification using fuzzy ARTMAP

The information available in languages other than English on the World Wide Web and global information systems is increasing significantly. Different languages can be produced by using one particular script such as Arabic, Persian, Urdu and Pashto that use Arabic script letters. The issue is how to produce reliable features of a web page that is to undergo language identification. Incorrectly identifying the language results in garbled translations as well as faulty and incomplete analyses. The aim of this study is to enhance the effectiveness of feature selection method of web page language identification. We have investigated total N-grams, N-grams frequency, N-grams frequency document frequency, and N-grams frequency inverse document frequency of web page language identification. From the experimental results, it is proven that N-grams frequency gives the most promising result compared to other feature selection methods.