Improved Feature Selection Research Articles

Explainable Machine Learning for Default Privacy Setting Prediction

When requesting a web-based service, users often fail in setting the website’s privacy settings according to their self privacy preferences. Being overwhelmed by the choice of preferences, a lack of knowledge of related technologies or unawareness of the own privacy preferences are just some reasons why users tend to struggle. To address all these problems, privacy setting prediction tools are particularly well-suited. Such tools aim to lower the burden to set privacy preferences according to owners’ privacy preferences. To be in line with the increased demand for explainability and interpretability by regulatory obligations – such as the General Data Protection Regulation (GDPR) in Europe – in this paper an explainable model for default privacy setting prediction is introduced. Compared to the previous work we present an improved feature selection, increased interpretability of each step in model design and enhanced evaluation metrics to better identify weaknesses in the model’s design before it goes into production. As a result, we aim to provide an explainable and transparent tool for default privacy setting prediction which users easily understand and are therefore more likely to use.

Improved Feature Selection Method for the Identification of Soil Images Using Oscillating Spider Monkey Optimization

Precision agriculture is the process that uses information and communication technology for farming and cultivation to improve overall productivity, efficient utilization of resources. Soil prediction is one of the primary phases in precision agriculture, resulting in good quality crops. In general, farmers perform the soil prediction manually. However, the efficiency of soil prediction may be enhanced by the use of current digital technologies. One effective way to automate soil prediction is image processing techniques in which soil images may be analyzed to determine the soil. This paper presents an efficient image analysis technique to predict the soil. For the same, a robust feature selection technique has been incorporated in the image analysis of soil images. The developed feature selection technique uses a new oscillating spider monkey optimization algorithm (OSMO) for the selection of features that are relevant and non-redundant. The new oscillating spider monkey optimization algorithm uses an oscillating perturbation rate to increase its precision and convergence behavior. A set of standard benchmark functions was deployed to visualize the performance of the new optimization technique (OSMO), and results were compared based on mean and standard deviation. Furthermore, the soil prediction approach is validated on a soil dataset, having seven categories. The proposed feature selection method selects the 41% relevant features, which provide the highest accuracy of 82.25% with 2.85% increase.

A gray wolf algorithm for feature and parameter selection of support vector classification

In classification problems, there are many data that contain a large number of features, some of which are irrelevant and cause confusion for the classifiers. The support vector classification (SVC) method is one of the most common methods used in classification. Feature selection, together with the parameters setting of SVC, such as the kernel parameter and the penalty parameter, significantly affects the classification performance of the SVC. In this study, the gray wolf optimisation (GWO) algorithm is proposed to improve feature selection and determine the optimal parameter values of SVC simultaneously. Based on several benchmark datasets for diseases, the experimental results show that the proposed method, FOGWO-SVC, is capable in selecting the best features with best parameters determination. Further, the comparative results demonstrate that the FOGWO-SVC is better or comparable than other competitor algorithms in terms of classification accuracy and feature reduction.

An improve feature selection algorithm for defect detection of glass bottles

Defect detection is an effective technology to guarantee the quality of the products. In this paper, the defect detection of glass bottles is taken as a classification problem. Features abstracted from the knocking sound signals of the glass bottles are used as the input of the classifier. Because the detection of bottles usually should be done in real time, feature selection which could reduce the dimensionality of data and simplify the learned model has become an important technology in this work. Based on our previous proposed algorithm named Shuffled Frog Leaping Algorithm-Improved Minimal Redundancy Maximal Relevance(SFLA-ImRMR), this paper focuses on improving the classification performance and proposes an improved feature selection algorithm. In the proposed algorithm, feature selection is combined with the training of the classifier, the wrapper approach is used to evaluate the selected features, Shuffled Frog Leaping Algorithm(SFLA) is used as the search algorithm, and BP Neural Network(BPNN) is used as the classifier. The proposed algorithm is named SFLA-ImRMR-BP and tested in the data sets built by the knocking sound signals of the glass bottles. Compared with SFLA-ImRMR and some other feature selection algorithms based on Evolutionary Computation(EC), features selected by SFLA-ImRMR-BP can achieve the highest classification performance.

Application of Fuzzy Entropy to Improve Feature Selection for Defect Recognition Using Support Vector Machine in High Voltage Cable Joints

This study presents a method for defect-recognition in high voltage cable joints based on partial discharge (PD). This recognition involves three major systematic procedures. In the first procedure, the PD patterns are produced by two different laboratory models representing two types of defects in a high voltage cable. The PD data are collected from a set of experiments in the PD tests with six high voltage cable joints, including prefabricated artificial defects. The second part involves feature selection by employing a fuzzy entropy algorithm by which the entropy value of each defect is computed. Using this fuzzy entropy algorithm, the features that have the most useful characteristics for distinguishing the defects in cable joints are found. In the third part, the selected features are used for testing and training the support vector machine (SVM) model, and the accuracy testing rates are calculated in order to obtain optimal results. The SVM model in this study achieves a higher accuracy rate of 96% for classification with the proposed feature-selection-based fuzzy entropy algorithm.

A novel method of constrained feature selection by the measurement of pairwise constraints uncertainty

In the past decades, the rapid growth of computer and database technologies has led to the rapid growth of large-scale datasets. On the other hand, data mining applications with high dimensional datasets that require high speed and accuracy are rapidly increasing. Semi-supervised learning is a class of machine learning in which unlabeled data and labeled data are used simultaneously to improve feature selection. The goal of feature selection over partially labeled data (semi-supervised feature selection) is to choose a subset of available features with the lowest redundancy with each other and the highest relevancy to the target class, which is the same objective as the feature selection over entirely labeled data. This method actually used the classification to reduce ambiguity in the range of values. First, the similarity values of each pair are collected, and then these values are divided into intervals, and the average of each interval is determined. In the next step, for each interval, the number of pairs in this range is counted. Finally, by using the strength and similarity matrices, a new constraint feature selection ranking is proposed. The performance of the presented method was compared to the performance of the state-of-the-art, and well-known semi-supervised feature selection approaches on eight datasets. The results indicate that the proposed approach improves previous related approaches with respect to the accuracy of the constrained score. In particular, the numerical results showed that the presented approach improved the classification accuracy by about 3% and reduced the number of selected features by 1%. Consequently, it can be said that the proposed method has reduced the computational complexity of the machine learning algorithm despite increasing the classification accuracy.

Wool knitted fabric pilling objective evaluation based on double-branch convolutional neural network

In the objective evaluation of wool knitted fabric pilling, the feature extraction step is a key factor affecting performance. In this paper, we proposed a double-branch deep cross-level fusion convolutional neural network (D-DCFNet) to improve feature selection. First, a cross-level fusion module (CLF module), based on a Fire module in SqueezeNet, was created to improve the feature extraction capability of a single module. Then, we designed a double-branch structure D-DCFNet. One branch consists of a CLF module, the core feature extraction module, and the other branch consists of a Fire module. Next, the features extracted from the two branches were fused together. Finally, the model trained by D-DCFNet was used to classify the knitting pilling data set to evaluate the robustness of the model. Experiments showed that D-DCFNet’s rating accuracy for woolen knitted fabrics and semi-worsted knitted fabrics is 99.35% and 99.02%, respectively, when the model size is only 5.77 M.

A consensus multi-view multi-objective gene selection approach for improved sample classification

BackgroundIn the field of computational biology, analyzing complex data helps to extract relevant biological information. Sample classification of gene expression data is one such popular bio-data analysis technique. However, the presence of a large number of irrelevant/redundant genes in expression data makes a sample classification algorithm working inefficiently. Feature selection is one such high-dimensionality reduction technique that helps to maximize the effectiveness of any sample classification algorithm. Recent advances in biotechnology have improved the biological data to include multi-modal or multiple views. Different ‘omics’ resources capture various equally important biological properties of entities. However, most of the existing feature selection methodologies are biased towards considering only one out of multiple biological resources. Consequently, some crucial aspects of available biological knowledge may get ignored, which could further improve feature selection efficiency.ResultsIn this present work, we have proposed a Consensus Multi-View Multi-objective Clustering-based feature selection algorithm called CMVMC. Three controlled genomic and proteomic resources like gene expression, Gene Ontology (GO), and protein-protein interaction network (PPIN) are utilized to build two independent views. The concept of multi-objective consensus clustering has been applied within our proposed gene selection method to satisfy both incorporated views. Gene expression data sets of Multiple tissues and Yeast from two different organisms (Homo Sapiens and Saccharomyces cerevisiae, respectively) are chosen for experimental purposes. As the end-product of CMVMC, a reduced set of relevant and non-redundant genes are found for each chosen data set. These genes finally participate in an effective sample classification.ConclusionsThe experimental study on chosen data sets shows that our proposed feature-selection method improves the sample classification accuracy and reduces the gene-space up to a significant level. In the case of Multiple Tissues data set, CMVMC reduces the number of genes (features) from 5565 to 41, with 92.73% of sample classification accuracy. For Yeast data set, the number of genes got reduced to 10 from 2884, with 95.84% sample classification accuracy. Two internal cluster validity indices - Silhouette and Davies-Bouldin (DB) and one external validity index Classification Accuracy (CA) are chosen for comparative study. Reported results are further validated through well-known biological significance test and visualization tool.

A new approach to health condition identification of rolling bearing using hierarchical dispersion entropy and improved Laplacian score

Since bearing fault signal under complex running status is usually manifested as the characteristics of nonlinear and non-stationary, which implies it is difficult to extract accurate affluent features and achieve effective fault identification via conventional signal processing tools. In this article, a hybrid intelligent fault identification scheme, the combination of hierarchical dispersion entropy and improved Laplacian score, is proposed to address this problem, which is mainly composed of three procedures. First, the particle swarm optimization–based optimized hierarchical dispersion entropy is adopted to excavate multilevel fault symptoms from low-frequency and high-frequency components, which can both solve the shortcoming of missing of high-frequency feature information existing in the recently presented multiscale dispersion entropy and artificial parameter selection issue of hierarchical dispersion entropy. Second, an improved feature selection strategy based on improved Laplacian score is proposed to select the sensitive features to establish a low-dimensional feature data set by incorporating the weight coefficient into Laplacian score. Finally, the established low-dimensional feature data set is fed to a Softmax classifier to automatically identify different health conditions of rolling bearing. The efficacy of the proposed method is validated by two experimental cases. Results show that our approach is highly effective in recognizing different fault categories and severities of rolling bearing. Meanwhile, our approach exhibits higher accuracy and better identification performance than some similar entropy-based hybrid approaches and other identification methods reported in this article.

Learning robust affinity graph representation for multi-view clustering

Recently, an increasingly pervasive trend in real-word applications is that the data are collected from multiple sources or represented by multiple views. Owing to the powerful ability of affinity graph in capturing the structural relationships among samples, constructing a robust and meaningful affinity graph has been extensively studied, especially in spectral clustering tasks. However, conventional spectral clustering extended to multi-view scenarios cannot obtain the satisfactory performance due to the presence of noise and the heterogeneity among different views. In this paper, we propose a robust affinity graph learning framework to deal with this issue. First, we employ an improved feature selection algorithm that integrates the advantages of hypergraph embedding and sparse regression to select significant features such that more robust graph Laplacian matrices for various views on this basis can be constructed. Second, we model hypergraph Laplacians as points on a Grassmann manifold and propose a Consistent Affinity Graph Learning (CAGL) algorithm to fuse all views. CAGL aims to learn a latent common affinity matrix shared by all Laplacian matrices by taking both the clustering quality evaluation criterion and the view consistency loss into account. We also develop an alternating descent algorithm to optimize the objective function of CAGL. Experiments on five publicly available datasets demonstrate that our proposed method obtains promising results compared with state-of-the-art methods.

Modelling an Effectual Glowworm Swarm Optimization Strategy for Feature Selection in Heart Disease Prediction

Heart disease is measured as a common disease all over the world. The ultimate target is to provide heart disease diagnosis with improved feature selection with Glow worm swarm optimization algorithm. The anticipated model comprises of optimization approach for feature selection and classifier for predicting heart disease. This system framework comprises of three stages: 1) data processing, 2) feature selection using IGWSO approach and classification with conventional machine learning classifiers. Here, C4.5 classifier is considered for performing the function. The benchmark dataset that has been attained from UCI database was cast off for performing computation. Maximal classification accuracy has been achieved based on cross validation strategy. Outcomes depicts that performance of anticipated model is superior in contrary to other models. Simulation has been done with MATLAB environment. Metrics like accuracy, sensitivity, specificity, F-measure and recall has been evaluated.

Enhancing BCI-Based Emotion Recognition Using an Improved Particle Swarm Optimization for Feature Selection.

Electroencephalogram (EEG) signals have been widely used in emotion recognition. However, the current EEG-based emotion recognition has low accuracy of emotion classification, and its real-time application is limited. In order to address these issues, in this paper, we proposed an improved feature selection algorithm to recognize subjects’ emotion states based on EEG signal, and combined this feature selection method to design an online emotion recognition brain-computer interface (BCI) system. Specifically, first, different dimensional features from the time-domain, frequency domain, and time-frequency domain were extracted. Then, a modified particle swarm optimization (PSO) method with multi-stage linearly-decreasing inertia weight (MLDW) was purposed for feature selection. The MLDW algorithm can be used to easily refine the process of decreasing the inertia weight. Finally, the emotion types were classified by the support vector machine classifier. We extracted different features from the EEG data in the DEAP data set collected by 32 subjects to perform two offline experiments. Our results showed that the average accuracy of four-class emotion recognition reached 76.67%. Compared with the latest benchmark, our proposed MLDW-PSO feature selection improves the accuracy of EEG-based emotion recognition. To further validate the efficiency of the MLDW-PSO feature selection method, we developed an online two-class emotion recognition system evoked by Chinese videos, which achieved good performance for 10 healthy subjects with an average accuracy of 89.5%. The effectiveness of our method was thus demonstrated.

Feature selection via normative fuzzy information weight with application into tumor classification

Feature selection via mutual information has been widely used in data analysing. Mutual information with monotonous is an effective tool to analyse the correlation and redundancy of features. However, the mutual information, which adopted in most of existing feature selection criterions, can’t explain the correlation and redundancy of features in the fuzzy situation well. Therefore, we propose feature selection strategy via normative fuzzy information weight based on fuzzy conditional mutual information in this paper. Firstly, the monotone fuzzy metric structure is defined, and some theoretical properties are proved. Secondly, we put forward the concept of fuzzy independent classification information based on fuzzy conditional mutual information, and propose a feature selection method via fuzzy independent classification information. Thirdly, considering the proportion of new classification information provided by the selected feature in its own information, we introduce the concept of normative fuzzy information weight and propose an improved feature selection method. Finally, the availability of the two proposed methods is tested by comparative experiments, and the improved feature selection method is applied to tumor classification. This work provides an alternative strategy for feature selection in real-world data applications.

Sparse multiple co-Inertia analysis with application to integrative analysis of multi -Omics data

BackgroundMultiple co-inertia analysis (mCIA) is a multivariate analysis method that can assess relationships and trends in multiple datasets. Recently it has been used for integrative analysis of multiple high-dimensional -omics datasets. However, its estimated loading vectors are non-sparse, which presents challenges for identifying important features and interpreting analysis results. We propose two new mCIA methods: 1) a sparse mCIA method that produces sparse loading estimates and 2) a structured sparse mCIA method that further enables incorporation of structural information among variables such as those from functional genomics.ResultsOur extensive simulation studies demonstrate the superior performance of the sparse mCIA and structured sparse mCIA methods compared to the existing mCIA in terms of feature selection and estimation accuracy. Application to the integrative analysis of transcriptomics data and proteomics data from a cancer study identified biomarkers that are suggested in the literature related with cancer disease.ConclusionProposed sparse mCIA achieves simultaneous model estimation and feature selection and yields analysis results that are more interpretable than the existing mCIA. Furthermore, proposed structured sparse mCIA can effectively incorporate prior network information among genes, resulting in improved feature selection and enhanced interpretability.

E-Commerce data classification in the cloud environment based on bayesian algorithm

In the process of e-commerce transactions, a large amount of data will be generated, whose effective classification is one of current research hotspots. An improved feature selection method was proposed based on the characteristics of Bayesian classification algorithm. Due to the long training and testing time of modern large-scale data classification on a single computer, a data classification algorithm based on Naive Bayes was designed and implemented on the Hadoop distributed platform. The experimental results showed that the improved algorithm could effectively improve the accuracy of classification, and the designed parallel Bayesian data classification algorithm had high efficiency, which was suitable for the processing and analysis of massive data.

Improved Feature Selection Model for Big Data Analytics

Although there are many attempts to build an optimal model for feature selection in Big Data applications, the complex nature of processing such kind of data makes it still a big challenge. Accordingly, the data mining process may be obstructed due to the high dimensionality and complexity of huge data sets. For the most informative features and classification accuracy optimization, the feature selection process constitutes a mandatory pre-processing phase to reduce dataset dimensionality. The exhaustive search for the relevant features is time-consuming. In this paper, a new binary variant of the wrapper feature selection grey wolf optimization and particle swarm optimization is proposed. The K-nearest neighbor classifier with Euclidean separation matrices is used to find the optimal solutions. A tent chaotic map helps in avoiding the algorithm from locked to the local optima problem. The sigmoid function employed for converting the search space from a continuous vector to a binary one to be suitable to the problem of feature selection. Cross-validation K-fold is used to overcome the overfitting issue. A variety of comparisons have been made with well-known and common algorithms, such as the particle swarm optimization algorithm, and the grey wolf optimization algorithm. Twenty datasets are used for the experiments, and statistical analyses are conducted to approve the performance and the effectiveness and of the proposed model with measures like selected features ratio, classification accuracy, and computation time. The cumulative features picked through the twenty datasets were 196 out of 773, as opposed to 393 and 336 in the GWO and the PSO, respectively. The overall accuracy is 90% relative to other algorithms ' 81.6 and 86.8. The total processing time for all datasets equals 184.3 seconds, wherein GWO and PSO equal 272 and 245.6, respectively.

Prediction of Adverse Drug Reactions Using Improved Feature Selection and Modified Fuzzy based Variability Ratio Tuning for SVM Classifier

Prediction of Adverse Drug Reactions Using Improved Feature Selection and Modified Fuzzy based Variability Ratio Tuning for SVM Classifier

Forecasting Plantago pollen: improving feature selection through random forests, clustering, and Friedman tests

Predicting concentrations of pollen is of great importance both for patients and for public health institutions. In this paper, we present a forecasting approach which relies on data and makes no assumptions on the underlying phenomena affecting the plants and the pollination process. Machine learning is used to build a model and to select the most important variables for prediction. Through nonparametric hypothesis testing, we show how some variables are indeed more important than others and how the careful combination of these variables can lead to more accurate and parsimonious models which avoid the huge computational times of more complex models while outperforming them in terms of the precision of the forecasts. By increasing the richness of the selected variables based on the clustered Friedman importance ranks, prediction error is reduced from 4.57 to 4.40 grains/m3 as an average, which accounts for a 3.5% average improvement across locations studied with a 50% reduction of execution times.

A Feature Selection Method based on the Pearson’s Correlation and Transformed Divergence Analysis

An improved feature selection method has been presented, which is based on Transformed Divergence (TD) considering weights of classes and Pearson’s correlation analysis. Using the improved method, this study evaluated several derived vegetation indices and texture measures based on Landsat-8 OLI data to determine their effect on improving the land cover classification separability in Jiangle county, Sanming city of Fujian province, China. The best vegetation indices combination was selected by the improved feature selection method and likewise, best textural combinations at different spatial resolution levels from multi-spectral bands or panchromatic band were obtained. The improved feature selection method found that a single feature could not maximize the separability of vegetation classes. When selecting four vegetation indices, the separability of vegetation classes can be maxmized significantly; and two textural measures were suitable for maxmizing the separability of vegetation classes. Overall, the result verifies that the feature selection method considering weights of classes and Pearson’s correlation coefficient can select optimal features to maximize class separability.

An Improved Feature Selection Method for Short Text Classification

Text has become one of the widest means of communication on mobile devices due to cheap rate and convenience for instance short text, web document, emails, instant messages. The exponential growth of text documents shared among users globally has increased the threat of misclassification associated with mobile devices such as Spam, Phishing, License to kill, Malware and privacy issues. Existing studies have shown that the major problem associated with text message classification is the poor representation of feature thus reducing accuracy and increasing f-measure rate. Thus, a modified Genetic Algorithm (GA) for improve feature selection and Artificial Immune System (AIS) algorithm was proposed for effective text classification in mobile short messages. The system will be deployed on an Android OS.