Filter Feature Selection Methods Research Articles

NEW STRATEGIES FOR IMPROVING NETWORK SECURITY AGAINST CYBER ATTACK BASED ON INTELLIGENT ALGORITHMS

Gradually, since the number of linked computer systems that use networks linked to the Internet is raised the information that is delivered through those systems becomes more vulnerable to cyber threats. This article presents proposed algorithms based on Machine Learning (ML) that ensure early detection of cyber threats that cause network breaking through the use of the Correlation Ranking Filter feature selection method. These proposed algorithms were applied to the Multi-Step Cyber-Attack Dataset (MSCAD) which consists of 66 features. The proposed strategy will apply machine learning algorithms like Adaptive Boosting-Deep Learning (AdaBoost-Deep Learning) or (ABDL), Multi-Layer Perceptron (MLP), Bayesian Networks Model (BNM), and Random Forest (RF), the feature would be decreased to high valuable of 46 features were included with a threshold of 0.1 or higher. The accuracy would be increased when the no. of features decreased to 46 with a threshold of ≥ 0.1 with the ABDL algorithm producing an accuracy of 99.7076%. The obtained results showed that the proposed algorithm delivered a suitable accuracy of 99.6791% with the ABDL algorithm even with a higher number of features.

A simple and efficient filter feature selection method via document-term matrix unitization

Text processing tasks commonly grapple with the challenge of high dimensionality. One of the most effective solutions to this challenge is to preprocess text data through feature selection methods. Feature selection can select the most advantageous features for subsequent operations (e.g., classification) from the native feature space of the text. This process effectively trims the feature space’s dimensionality, enhancing subsequent operations’ efficiency and accuracy. This paper proposes a straightforward and efficient filter feature selection method based on document-term matrix unitization (DTMU) for text processing. Diverging from previous filter feature selection methods that concentrate on scoring criteria definition, our method achieves more optimal feature selection by unitizing each column of the document-term matrix. This approach mitigates feature-to-feature influence and reinforces the role of the weighting proportion within the features. Subsequently, our scoring criterion subtracts the sum of weights for negative samples from positive samples and takes the absolute value. We conduct numerical experiments to compare DTMU with four advanced filter feature selection methods: max–min ratio metric, proportional rough feature selector, least loss, and relative discrimination criterion, along with two classical filter feature selection methods: Chi-square and information gain. The experiments are performed on four ten-thousand-dimensional feature space datasets: book, dvd, music, movie and two thousand-dimensional feature space datasets: imdb, amazon_cells, sourced from Amazon product reviews and movie reviews. Experimental findings demonstrate that DTMU selects more advantageous features for subsequent operations and achieves a higher dimensionality reduction rate than those of the other six methods used for comparison. Moreover, DTMU exhibits robust generalization capabilities across various classifiers and dimensional datasets. Notably, the average CPU time for a single run of DTMU is measured at 1.455 s.

A hybrid feature selection model for identifying groups of critical elements in aero‐engine assembly

AbstractAero‐engine assembly has a wide range of indicators, and finding critical assembly elements to guide assembly and troubleshooting is essential in improving the reliability and safety of aero‐engine. In order to identify critical elements in aero‐engine assembly components, this study aims to establish a two‐stage hybrid feature selection model, namely, FSBP approach, which integrated filter method and particle swarm algorithm with Bayesian optimization. Specifically, individual filter feature selection methods are compared to select a relatively effective method to reduce the data dimensions and ensure the quality of the initial subset. Then, the particle swarm algorithm combined with Bayesian optimization obtains a subset of features in the second stage that are more suitable for predictive models with more robust classification prediction capability. The algorithm is successfully applied to real aero‐engine assembly and trial test datasets, and the experimental results show that our proposed two‐stage hybrid feature selection model outperforms other benchmark methods.

The application of machine learning on brain imaging features of different narcolepsy subtypes.

Narcolepsy is a central hypersomnia disorder, and differential diagnoses between its subtypes can be difficult. Hence, we applied machine learning to analyze the positron emission tomography (PET) data of patients with type 1 or type 2 narcolepsy, and patients with type 1 narcolepsy and comorbid schizophrenia, to construct predictive models to facilitate the diagnosis. This is a retrospective and prospective case-control study of adolescent and young adult patients with type 1 or type 2 narcolepsy, and type 1 narcolepsy and comorbid schizophrenia. All participants received 18-F-fluorodeoxy glucose PET, sleep studies, neurocognitive tests, sleep questionnaires, and human leukocyte antigen typing. The collected PET data were analyzed by feature selections and classification methods in machine learning to construct predictive models. A total of 314 participants with narcolepsy were enrolled; 204 had type 1 narcolepsy, 90 had type 2 narcolepsy, and 20 had type 1 narcolepsy and comorbid schizophrenia. We used three filter methods for feature selection followed by a comparative analysis of classification methods. To apply a small number of regions of interest (ROI) and high classification accuracy, the Naïve Bayes classifier with the Term Variance as feature selection achieved the goal with only three ROIs (left basal ganglia, left Heschl, and left striatum) and produced an accuracy of higher than 99%. The accuracy of our predictive model of PET data are promising and can aid clinicians in the diagnosis of narcolepsy subtypes. Future research with a larger sample size could further refine the predictive model of narcolepsy.

Classification of Breast Cancer using Ensemble Filter Feature Selection with Triplet Attention Based Efficient Net Classifier

In medical imaging, the effective detection and classification of Breast Cancer (BC) is a current research important task because of the still existing difficulty to distinguish abnormalities from normal breast tissues due to their subtle appearance and ambiguous margins and distinguish abnormalities from the normal breast. Moreover, BC detection based on an automated detection model is needed, because manual diagnosis faces problems due to cost and shortage of skilled manpower, and also takes a very long time. Using deep learning and ensemble feature selection techniques, in this paper, a novel framework is introduced for classifying BC from histopathology images. The five primary steps of the suggested framework are as follows: 1) to make the largest original dataset and then deep learning model with data augmentation to improve the learning. 2) The best features are selected by an Ensemble Filter Feature selection Method (EFFM) which combines the best feature subsets to produce the final feature subsets. 3) Then the pruned Convolution Neural Network (CNN) model is utilized to extract the optimal features. 4) Finally, the classification is done through the Triplet Attention based Efficient Network (TAENet) classifier. The suggested model produces a 98% accuracy rate after being trained and tested on two different histopathology imaging datasets including images from four different data cohorts. Subsequently, the suggested strategy outperforms the conventional ones since the ensemble filter habitually acquires the best features, and experimental results demonstrate the importance of the proposed approach

Analyzing the impact of feature selection methods on machine learning algorithms for heart disease prediction

The present study examines the role of feature selection methods in optimizing machine learning algorithms for predicting heart disease. The Cleveland Heart disease dataset with sixteen feature selection techniques in three categories of filter, wrapper, and evolutionary were used. Then seven algorithms Bayes net, Naïve Bayes (BN), multivariate linear model (MLM), Support Vector Machine (SVM), logit boost, j48, and Random Forest were applied to identify the best models for heart disease prediction. Precision, F-measure, Specificity, Accuracy, Sensitivity, ROC area, and PRC were measured to compare feature selection methods' effect on prediction algorithms. The results demonstrate that feature selection resulted in significant improvements in model performance in some methods (e.g., j48), whereas it led to a decrease in model performance in other models (e.g. MLP, RF). SVM-based filtering methods have a best-fit accuracy of 85.5. In fact, in a best-case scenario, filtering methods result in + 2.3 model accuracy. SVM-CFS/information gain/Symmetrical uncertainty methods have the highest improvement in this index. The filter feature selection methods with the highest number of features selected outperformed other methods in terms of models' ACC, Precision, and F-measures. However, wrapper-based and evolutionary algorithms improved models' performance from sensitivity and specificity points of view.

Machine learning model for predicting immediate postoperative desaturation using spirometry signal data

Postoperative desaturation is a common post-surgery pulmonary complication. The real-time prediction of postoperative desaturation can become a preventive measure, and real-time changes in spirometry data can provide valuable information on respiratory mechanics. However, there is a lack of related research, specifically on using spirometry signals as inputs to machine learning (ML) models. We developed an ML model and postoperative desaturation prediction index (DPI) by analyzing intraoperative spirometry signals in patients undergoing laparoscopic surgery. We analyzed spirometry data from patients who underwent laparoscopic, robot-assisted gynecologic, or urologic surgery, identifying postoperative desaturation as a peripheral arterial oxygen saturation level below 95%, despite facial oxygen mask usage. We fitted the ML model on two separate datasets collected during different periods. (Datasets A and B). Dataset A (Normal 133, Desaturation 74) was used for the entire experimental process, including ML model fitting, statistical analysis, and DPI determination. Dataset B (Normal 20, Desaturation 4) was only used for verify the ML model and DPI. Four feature categories—signal property, inter-/intra-position correlation, peak value/interval variability, and demographics—were incorporated into the ML models via filter and wrapper feature selection methods. In experiments, the ML model achieved an adequate predictive capacity for postoperative desaturation, and the performance of the DPI was unbiased.

A novel enterprise credit risk rating model with the perspective of multiple regulatory demands

With the number of market participants rising, the issue of contradiction between heavy supervision task and insufficient supervision resource have become increasingly urgent for regulatory authorities to consider. The credit risk rating of enterprises is therefore introduced to improve the efficiency of supervision. By analyzing the purposes of enterprise credit risk rating and the multiple demands for regulatory authorities, this paper offers a multi-classification ensemble model with hybrid decomposition strategy and two-stage multi-objective feature selection. First, an adaptive resampling technique is introduced into the hybrid decomposition strategy for multi-classification, so to appropriately allocate supervision resources in each credit risk class. Then, a two-stage multi-objective feature selection method is developed to combine the various demands of regulatory authorities for feature selection. In the first stage, results of multiple Filter feature selection methods are integrated as output to improve the performance of feature information; In the second stage, the feature information from the first stage is used as input to guide the Wrapper feature selection, which is based on the Improved Multi-objective Ant Colony Optimization. Finally, a real listed enterprise credit dataset of China is used to evaluate the performance of each module and the entire model. The empirical results show that the design of hybrid decomposition strategy and multi-objective feature selection can not only deal with the class-imbalanced problem during the decomposition process, but also take into account the balance of each demand well. More importantly, a desirable framework is set in this paper to simplify the process from imbalanced multi-classification to a balanced binary one.

Filter and Wrapper Stacking Ensemble (FWSE): a robust approach for reliable biomarker discovery in high-dimensional omics data.

Selecting informative features, such as accurate biomarkers for disease diagnosis, prognosis and response to treatment, is an essential task in the field of bioinformatics. Medical data often contain thousands of features and identifying potential biomarkers is challenging due to small number of samples in the data, method dependence and non-reproducibility. This paper proposes a novel ensemble feature selection method, named Filter and Wrapper Stacking Ensemble (FWSE), to identify reproducible biomarkers from high-dimensional omics data. In FWSE, filter feature selection methods are run on numerous subsets of the data to eliminate irrelevant features, and then wrapper feature selection methods are applied to rank the top features. The method was validated on four high-dimensional medical datasets related to mental illnesses and cancer. The results indicate that the features selected by FWSE are stable and statistically more significant than the ones obtained by existing methods while also demonstrating biological relevance. Furthermore, FWSE is a generic method, applicable to various high-dimensional datasets in the fields of machine intelligence and bioinformatics.

Filter feature selection methods for text classification: a review

Filter feature selection methods for text classification: a review

Research on Credit Risk Prediction under Unbalanced Dataset Based on Ensemble Learning

Aiming at the problem of high-dimensional features and data imbalance in credit risk assessment that affect model prediction results, a credit risk prediction method based on integrated learning is proposed from three levels of features, data and algorithms. First, a hybrid filter and Random Forest feature selection method is used to select features. This method uses the improved Relief algorithm to initially select features, then combines the maximum information coefficient to eliminate redundant features, and uses the Random Forest algorithm to further reduce the feature dimension. Second, on the basis of the Borderline-SMOTE method, an adaptive idea is introduced to generate a different number of new samples for each minority sample at the boundary, and a new interpolation method is used to make the new sample distribution more reasonable, so as to reduce the sample imbalance. Finally, the Focal Loss is used to improve the loss function of LightGBM, and the sample weight is adjusted through the parameters α and γ in the Focal Loss function so that the model pays more attention to minority samples and difficult-to-classify samples, and improves the accuracy of model classification. And use the improved algorithm as the base classifier and then use AdaBoost and random subspace methods to integrate to establish a credit risk prediction model. Through comparative experiments with other methods, the results show that this method effectively improves the <math xmlns="http://www.w3.org/1998/Math/MathML" id="M1"> <mi>G</mi> </math> -mean value and the AUC value and has a better default prediction effect.

A two-stage hybrid biomarker selection method based on ensemble filter and binary differential evolution incorporating binary African vultures optimization

BackgroundIn the field of genomics and personalized medicine, it is a key issue to find biomarkers directly related to the diagnosis of specific diseases from high-throughput gene microarray data. Feature selection technology can discover biomarkers with disease classification information.ResultsWe use support vector machines as classifiers and use the five-fold cross-validation average classification accuracy, recall, precision and F1 score as evaluation metrics to evaluate the identified biomarkers. Experimental results show classification accuracy above 0.93, recall above 0.92, precision above 0.91, and F1 score above 0.94 on eight microarray datasets.MethodThis paper proposes a two-stage hybrid biomarker selection method based on ensemble filter and binary differential evolution incorporating binary African vultures optimization (EF-BDBA), which can effectively reduce the dimension of microarray data and obtain optimal biomarkers. In the first stage, we propose an ensemble filter feature selection method. The method combines an improved fast correlation-based filter algorithm with Fisher score. obviously redundant and irrelevant features can be filtered out to initially reduce the dimensionality of the microarray data. In the second stage, the optimal feature subset is selected using an improved binary differential evolution incorporating an improved binary African vultures optimization algorithm. The African vultures optimization algorithm has excellent global optimization ability. It has not been systematically applied to feature selection problems, especially for gene microarray data. We combine it with a differential evolution algorithm to improve population diversity.ConclusionCompared with traditional feature selection methods and advanced hybrid methods, the proposed method achieves higher classification accuracy and identifies excellent biomarkers while retaining fewer features. The experimental results demonstrate the effectiveness and advancement of our proposed algorithmic model.

A Multi-level Random Forest Model-Based Intrusion Detection Using Fuzzy Inference System for Internet of Things Networks

Intrusion detection (ID) methods are security frameworks designed to safeguard network information systems. The strength of an intrusion detection method is dependent on the robustness of the feature selection method. This study developed a multi-level random forest algorithm for intrusion detection using a fuzzy inference system. The strengths of the filter and wrapper approaches are combined in this work to create a more advanced multi-level feature selection technique, which strengthens network security. The first stage of the multi-level feature selection is the filter method using a correlation-based feature selection to select essential features based on the multi-collinearity in the data. The correlation-based feature selection used a genetic search method to choose the best features from the feature set. The genetic search algorithm assesses the merits of each attribute, which then delivers the characteristics with the highest fitness values for selection. A rule assessment has also been used to determine whether two feature subsets have the same fitness value, which ultimately returns the feature subset with the fewest features. The second stage is a wrapper method based on the sequential forward selection method to further select top features based on the accuracy of the baseline classifier. The selected top features serve as input into the random forest algorithm for detecting intrusions. Finally, fuzzy logic was used to classify intrusions as either normal, low, medium, or high to reduce misclassification. When the developed intrusion method was compared to other existing models using the same dataset, the results revealed a higher accuracy, precision, sensitivity, specificity, and F1-score of 99.46%, 99.46%, 99.46%, 93.86%, and 99.46%, respectively. The classification of attacks using the fuzzy inference system also indicates that the developed method can correctly classify attacks with reduced misclassification. The use of a multi-level feature selection method to leverage the advantages of filter and wrapper feature selection methods and fuzzy logic for intrusion classification makes this study unique.

Identifying Diagnostic Biomarkers of Breast Cancer Based on Gene Expression Data and Ensemble Feature Selection

Background: In recent years, the identification of biomarkers or signatures based on gene expression profiling data has attracted much attention in bioinformatics. The successful discovery of breast cancer (BRCA) biomarkers will be beneficial in reducing the risk of BRCA among patients for early detection. Methods: This paper proposes an Ensemble Feature Selection method to screen biomarkers (abbreviat-ed as EFSmarker) for BRCA from publically available gene expression data. Firstly, we employ twelve filter feature selection methods, namely median, variance, Chi-square, Relief, Pearson and Spearman correlation, mutual information, minimal-redundancy-maximal-relevance criterion, ridge regression, decision tree and random forest with Gini index and accuracy index, to calculate the importance (weights or coefficients) of all features on the training dataset. Secondly, we apply the logistic regres-sion classifier on the test dataset to calculate the classification AUC value of each feature subset indi-vidually selected by twelve methods. Thirdly, we provide an ensemble feature selection method by ag-gregating feature importance with classification AUC value. In particular, we establish a feature im-portance score (FIS) to evaluate the importance of each feature underlying all feature selection methods. Finally, the features with higher FIS are taken as identified biomarkers. Results: With the direction of the FIS index induced by the EFSmarker method, 12 genes (COL10A1, COL11A1, MMP11, LOC728264, FIGF, GJB2, INHBA, CD300LG, IGFBP6, PAMR1, CXCL2 and FXYD1) are regarded as diagnostic biomarkers for BRCA. Especially, COL10A1, ranked first with a FIS value of 0.663, is identified as the most credible biomarker. The findings justified via gene and protein expression validation, functional enrichment analysis, literature checking and independent dataset validation verify the effectiveness and efficiency of these selected biomarkers. Conclusion: Our proposed biomarker discovery strategy not only utilizes the feature contribution but also considers the prediction accuracy simultaneously, which may also serve as a model for identifying unknown biomarkers for other diseases from high-throughput gene expression data. The source code and data are available at https://github.com/zpliulab/EFSmarker.

Hybrid Filter and Genetic Algorithm-Based Feature Selection for Improving Cancer Classification in High-Dimensional Microarray Data

The advancements in intelligent systems have contributed tremendously to the fields of bioinformatics, health, and medicine. Intelligent classification and prediction techniques have been used in studying microarray datasets, which store information about the ways used to express the genes, to assist greatly in diagnosing chronic diseases, such as cancer in its earlier stage, which is important and challenging. However, the high-dimensionality and noisy nature of the microarray data lead to slow performance and low cancer classification accuracy while using machine learning techniques. In this paper, a hybrid filter-genetic feature selection approach has been proposed to solve the high-dimensional microarray datasets problem which ultimately enhances the performance of cancer classification precision. First, the filter feature selection methods including information gain, information gain ratio, and Chi-squared are applied in this study to select the most significant features of cancerous microarray datasets. Then, a genetic algorithm has been employed to further optimize and enhance the selected features in order to improve the proposed method’s capability for cancer classification. To test the proficiency of the proposed scheme, four cancerous microarray datasets were used in the study—this primarily included breast, lung, central nervous system, and brain cancer datasets. The experimental results show that the proposed hybrid filter-genetic feature selection approach achieved better performance of several common machine learning methods in terms of Accuracy, Recall, Precision, and F-measure.

Heterogeneous Ensemble Feature Selection for Network Intrusion Detection System

Intrusion detection systems get more attention to secure the computers and network systems. Researchers propose different network intrusion detection systems using machine learning techniques. However, the massive amount of data that contain irrelevant and redundant features is still challenging the intrusion detection systems. The redundancy and irrelevance of features may slow the processing time and decrease prediction performance. This paper proposes a Heterogeneous Ensemble Feature Selection (HEFS) method to select the relevant features while achieving better attack detection performance. The proposed method fuses the output feature subsets of five filter feature selection methods, using a union combination method, to obtain an ensemble features subset. HEFS method uses merit-based evaluation to avoid the internal redundancy of the obtained ensemble features subset and acquire the final optimal features. We evaluate the HEFS method with random forest, J48, random tree, and REP tree. In a multi-class NSL-KDD dataset, the experimental results show that the proposed method achieves better prediction performance than the specific feature selection methods and other frameworks.

A filter feature selection for high-dimensional data

In a classification problem, before building a prediction model, it is very important to identify informative features rather than using tens or thousands which may penalize some learning methods and increase the risk of over-fitting. To overcome these problems, the best solution is to use feature selection. In this article, we propose a new filter method for feature selection, by combining the Relief filter algorithm and the multi-criteria decision-making method called TOPSIS (Technique for Order Preference by Similarity to Ideal Solution), we modeled the feature selection task as a multi-criteria decision problem. Exploiting the Relief methodology, a decision matrix is computed and delivered to Technique for Order Preference by Similarity to Ideal Solution in order to rank the features. The proposed method ends up giving a ranking to the features from the best to the mediocre. To evaluate the performances of the suggested approach, a simulation study including a set of experiments and case studies was conducted on three synthetic dataset scenarios. Finally, the obtained results approve the effectiveness of our proposed filter to detect the best informative features.

JoMIC: A joint MI-based filter feature selection method

Feature selection (FS) is a common preprocessing step of machine learning that selects informative subset of features which fuels a model to perform better during prediction or classification. It helps in the design of an intelligent and expert system used in computer vision, image processing, gene expression data analysis, intrusion detection and natural language processing. In this paper, we introduce an effective filter method called Joint Mutual Information with Class relevance (JoMIC) using multivariate Joint Mutual Information (JMI) and Mutual Information (MI). Our method considers both JMI and MI of a non selected feature with selected ones w.r.t a given class to select a feature that is highly relevant to the class but non redundant to other selected features. We compare our method with seven other filter-based methods using the machine learning classifiers viz., Logistic Regression, Support Vector Machine, K-nearest Neighbor (KNN), Decision Tree, Random Forest, Naïve Bayes, and Stochastic Gradient Descent on various datasets. Experimental results reveal that our method yields better performance in terms of accuracy, Matthew’s Correlation Coefficient (MCC) and F1-score over 16 benchmark datasets, as compared to other competent methods. The superiority of our proposed method is that it uses an effective objective function that combines both JMI and MI to choose the relevant and non redundant features.

A Multi-label Filter Feature Selection Method Based on Approximate Pareto Dominance

A Multi-label Filter Feature Selection Method Based on Approximate Pareto Dominance

Predicting perinatal mortality based on maternal health status and health insurance service using homogeneous ensemble machine learning methods

BackgroundPerinatal mortality in Ethiopia is the highest in Africa, with 68 per 1000 pregnancies intrapartum deaths. It is mainly associated with home delivery, which contributes to more than 75% of perinatal deaths. Financial constraints significantly impact timely access to maternal health care. Financial incentives, such as health insurance, may address the demand- and supply-side factors. This study, hence, aims to predict perinatal mortality based on maternal health status and health insurance service using homogeneous ensemble machine learning methods.MethodsThe data was collected from the Ethiopian demographic health survey from 2011 to 2019 G.C. The data were pre-processed to get quality data that are suitable for the homogenous ensemble machine-learning algorithms to develop a model that predicts perinatal mortality. We have applied filter (chi-square and mutual information) and wrapper (sequential forward and sequential backward) feature selection methods. After selecting all the relevant features, we developed a predictive model using cat boost, random forest, and gradient boosting algorithms and evaluated the model using both objective (accuracy, precision, recall, F1_score, ROC) and subjective (domain expert) based evaluation techniques.ResultsPerinatal mortality prediction models were developed using random forest, gradient boosting, and cat boost algorithms with the overall accuracy of 89.95%, 90.24%, and 82%, respectively. Risk factors of perinatal mortality were identified using feature importance analysis and relevant rules were extracted using the best performing model.ConclusionsA prediction model that was developed using gradient boosting algorithms was selected for further use in the risk factor analysis, generating relevant rules, development of artifacts, and model deployment because it has registered better performance with 90.24% accuracy. The most determinant risk factors of perinatal mortality were identified using feature importance and some of them are community-based health insurance, mother's educational level, region and place of residence, age, wealth status, birth interval, preterm, smoking cigarette, anemia level, hemoglobin level, and marital status.