Search Strategies For Feature Selection Research Articles

MAFSIDS: a reinforcement learning-based intrusion detection model for multi-agent feature selection networks

Large unbalanced datasets pose challenges for machine learning models, as redundant and irrelevant features can hinder their effectiveness. Furthermore, the performance of intrusion detection systems (IDS) can be further degraded by the emergence of new network attack types. To address these issues, we propose MAFSIDS (Multi-Agent Feature Selection Intrusion Detection System), a DQL (Deep Q-Learning) based IDS.MAFSIDS comprises a feature self-selection algorithm and a DRL (Deep Reinforcement Learning) attack detection module. The feature self-selection algorithm leverages a multi-agent reinforcement learning framework, which redefines the feature selection problem by converting the traditional {2}^{N} feature selection space into N agent representations. This approach reduces model complexity and enhances the search strategy for feature selection. To ensure accurate feature representation and expedite the feature selection process, we have also developed a GCN (Graph Convolutional Network) method that extracts deeper features from the data. The DRL attack detection module utilizes the Mini-Batchs technique to encode the data, allowing reinforcement learning to be applied in a supervised learning context. This integration improves accuracy. Additionally, the policy network in this module is designed to be minimalist, enhancing model efficiency. To evaluate the performance of our model, we conducted comprehensive simulation experiments using Python. We tested the model using the CSE-CIC-IDS2018 and NSL-KDD datasets, achieving impressive accuracy rates of 96.8% and 99.1%, as well as F1-Scores of 96.3% and 99.1%, respectively. The selected feature subset successfully eliminates approximately 80% of redundant features compared to the original feature set. Furthermore, we compared our proposed model with other popular machine-learning models.

Improved Equilibrium Optimization Algorithm Using Elite Opposition-Based Learning and New Local Search Strategy for Feature Selection in Medical Datasets

The rapid growth in biomedical datasets has generated high dimensionality features that negatively impact machine learning classifiers. In machine learning, feature selection (FS) is an essential process for selecting the most significant features and reducing redundant and irrelevant features. In this study, an equilibrium optimization algorithm (EOA) is used to minimize the selected features from high-dimensional medical datasets. EOA is a novel metaheuristic physics-based algorithm and newly proposed to deal with unimodal, multi-modal, and engineering problems. EOA is considered as one of the most powerful, fast, and best performing population-based optimization algorithms. However, EOA suffers from local optima and population diversity when dealing with high dimensionality features, such as in biomedical datasets. In order to overcome these limitations and adapt EOA to solve feature selection problems, a novel metaheuristic optimizer, the so-called improved equilibrium optimization algorithm (IEOA), is proposed. Two main improvements are included in the IEOA: The first improvement is applying elite opposite-based learning (EOBL) to improve population diversity. The second improvement is integrating three novel local search strategies to prevent it from becoming stuck in local optima. The local search strategies applied to enhance local search capabilities depend on three approaches: mutation search, mutation–neighborhood search, and a backup strategy. The IEOA has enhanced the population diversity, classification accuracy, and selected features, and increased the convergence speed rate. To evaluate the performance of IEOA, we conducted experiments on 21 biomedical benchmark datasets gathered from the UCI repository. Four standard metrics were used to test and evaluate IEOA’s performance: the number of selected features, classification accuracy, fitness value, and p-value statistical test. Moreover, the proposed IEOA was compared with the original EOA and other well-known optimization algorithms. Based on the experimental results, IEOA confirmed its better performance in comparison to the original EOA and the other optimization algorithms, for the majority of the used datasets.

Optimized grass hopper algorithm for diagnosis of Parkinson’s disease

The Modified Grasshopper Optimization Algorithm which identifies Parkinson disease symptoms at an early (premature) stage was proposed. Parkinson disease, type of movement ailment, could be life-threatening if not treated at premature stage. Therefore, diagnosis of Parkinson disease became essential in early stages so that all the symptoms could be controlled by giving required medication to the patient. Hence ensuring the patient longevity. As part of this research work, a novel model Modified Grasshopper Optimization Algorithm was introduced which was based on the traditional Grasshopper Optimization Algorithm and search strategy for feature selection. Grasshopper Optimization Algorithm was relatively a novel heuristic optimization swarm intelligence algorithm which was stimulated by grasshoppers searching for food. This population-based method has capability to provide solution for real-life problems in undefined search space. It mimics grasshopper swarm’s behaviour and their social interaction. Popular algorithms like Random Forest, Decision Tree and k-Nearest Neighbour classifier were used in judgement on shortlisted aka selected features. Different datasets of handwriting (meander and spiral), speech and voice were used for evaluating the presented model. The proposed algorithm was effective in Parkinson disease identification having accuracy (computed) of 95.37%, 99.47% detection rate and 15.78% false alarm rate. This helps larger cause of patient in receiving treatment in pre-mature stage. The presented bio-inspired algorithm was adequately steady and has ability to identify the optimal feature set. Finally results obtained from the assessment of introduced Modified Grasshopper Optimization Algorithm on these data sets were evaluated and contrasted with respect to outcome of Modified Grey Wolf Optimizer and Optimized Cuttlefish Algorithm. The experiment’s outcome revealed that the presented Modified Grasshopper Optimization Algorithm assists in reducing the selected features count and improving the accuracy.

Diagnosis of Parkinson’s disease using modified grey wolf optimization

This paper presents the Modified Grey Wolf Optimization (MGWO) algorithm which helps with the identification of the symptoms of Parkinson’s disease at a premature stage. Parkinson disease is kind of a movement malady, which if not cured timely can prove to be fatal.Thus it becomes significant to identify Parkinson’s disease at its premature phase so proper medications can provide longevity to patient by controlling the symptoms. In this work, a new model named Modified Grey Wolf Optimization (MGWO) has been proposed grounded on the traditional Grey Wolf Optimizer (GWO), which acts as a search strategy for feature selection. GWO is a meta-heuristic algorithm which is enthused by hunt down behavior of wolves. Random forest, k-nearest neighbor classifier and decision tree espy on selected features. The proposed model is evaluated using various types of datasets of voice, handwriting (spiral and meander) and speech. The put forward algorithm helps in the prediction of Parkinson disease with an estimated accuracy of 94.83%, detection rate of 98.28%, false alarm rate of 16.03% and further aid the individuals to receive a functional treatment at an early stage. The proposed bio-inspired algorithm is stable enough to find out the optimal subset of features. At last the results derived from the evaluation of proposed algorithm on datasets are compared with the results of Optimized Cuttlefish Algorithm (OCFA). The experimental results depict that the proposed algorithm helps in maximizing the accurateness and minimizing the number of features selected.

A methodology for evaluating multi-objective evolutionary feature selection for classification in the context of virtual screening

Virtual screening (VS) methods have been shown to increase success rates in many drug discovery campaigns, when they complement experimental approaches, such as high-throughput screening methods or classical medicinal chemistry approaches. Nevertheless, predictive capability of VS is not yet optimal, mainly due to limitations in the underlying physical principles describing drug binding phenomena. One approach that can improve VS methods is the aid of machine learning methods. When enough experimental data are available to train such methods, predictive capability can considerably increase. We show in this research work how a multi-objective evolutionary search strategy for feature selection, which can provide with small and accurate decision trees that can be very easily understood by chemists, can drastically increase the applicability and predictive ability of these techniques and therefore aid considerable in the drug discovery problem. With the proposed methodology, we find classification models with accuracy between 0.9934 and 1.00 and area under ROC between 0.96 and 1.00 evaluated in full training sets, and accuracy between 0.9849 and 0.9940 and area under ROC between 0.89 and 0.93 evaluated with tenfold cross-validation over 30 iterations, while substantially reducing the model size.

Unsupervised feature selection for interpretable classification in behavioral assessment of children

AbstractIn this paper, we consider a data set taken from the administration of the Behavior Assessment System for Children test to 157 subjects, and we approach the problem of clustering and classify the subjects in an interpretable fashion. Because the Behavior Assessment System for Children test is originally composed of 149 questions (152 in the particular version used for this experiment), we first propose a feature selection wrapper model composed by a multi‐objective evolutionary algorithm, the iterative clustering method expectation–maximization, and the classifier C4.5 for the unsupervised feature selection towards the classification of the data with two objectives: maximizing the likelihood of the clustering model and maximizing the accuracy of the obtained classifier. We propose a methodology to integrate feature selection for unsupervised classification, model evaluation, decision‐making (to choose the most satisfactory model according to an a posteriori process in a multi‐objective context), and testing. The selected data set that is the result of this process, where each instance is labeled with its class, is then used for supervised learning via both C4.5 and a novel evolutionary computation‐based fuzzy classifier to obtain interpretable rules. We discuss and compare the behavior of two different evolutionary algorithms (ENORA (Evolutionary NOn‐dominated Radial slots based Algorithm) and NSGA‐II (Non‐dominated Sorted Genetic Algorithm)) at different levels: as search strategies for feature selection, as search strategies for fuzzy classification, and in terms of quality of the results. It turns out that ENORA behaves better in terms of quality of the result in the feature selection phase (obtaining a selection that shows higher accuracy under C4.5 after cross‐validation), and again in the fuzzy classification phase, from both points of view: hypervolume evolution and interpretability of results. During the entire process, the solutions are validated by the psychologists who collected the data.

A Semidefinite Programming Based Search Strategy for Feature Selection with Mutual Information Measure.

Feature subset selection, as a special case of the general subset selection problem, has been the topic of a considerable number of studies due to the growing importance of data-mining applications. In the feature subset selection problem there are two main issues that need to be addressed: (i) Finding an appropriate measure function than can be fairly fast and robustly computed for high-dimensional data. (ii) A search strategy to optimize the measure over the subset space in a reasonable amount of time. In this article mutual information between features and class labels is considered to be the measure function. Two series expansions for mutual information are proposed, and it is shown that most heuristic criteria suggested in the literature are truncated approximations of these expansions. It is well-known that searching the whole subset space is an NP-hard problem. Here, instead of the conventional sequential search algorithms, we suggest a parallel search strategy based on semidefinite programming (SDP) that can search through the subset space in polynomial time. By exploiting the similarities between the proposed algorithm and an instance of the maximum-cut problem in graph theory, the approximation ratio of this algorithm is derived and is compared with the approximation ratio of the backward elimination method. The experiments show that it can be misleading to judge the quality of a measure solely based on the classification accuracy, without taking the effect of the non-optimum search strategy into account.

A Wearable Sensor Module With a Neural-Network-Based Activity Classification Algorithm for Daily Energy Expenditure Estimation

This paper presents a wearable module and neural-network-based activity classification algorithm for energy expenditure estimation. The purpose of our design is first to categorize physical activities with similar intensity levels, and then to construct energy expenditure regression (EER) models using neural networks in order to optimize the estimation performance. The classification of physical activities for EER model construction is based on the acceleration and ECG signal data collected by wearable sensor modules developed by our research lab. The proposed algorithm consists of procedures for data collection, data preprocessing, activity classification, feature selection, and construction of EER models using neural networks. In order to reduce the computational load and achieve satisfactory estimation performance, we employed sequential forward and backward search strategies for feature selection. Two representative neural networks, a radial basis function network (RBFN) and a generalized regression neural network (GRNN), were employed as EER models for performance comparisons. Our experimental results have successfully validated the effectiveness of our wearable sensor module and its neural-network-based activity classification algorithm for energy expenditure estimation. In addition, our results demonstrate the superior performance of GRNN as compared to RBFN.

A dependency-based search strategy for feature selection

Feature selection has become an increasingly important field of research. It aims at finding optimal feature subsets that can achieve better generalization on unseen data. However, this can be a very challenging task, especially when dealing with large feature sets. Hence, a search strategy is needed to explore a relatively small portion of the search space in order to find “semi-optimal” subsets. Many search strategies have been proposed in the literature, however most of them do not take into consideration relationships between features. Due to the fact that features usually have different degrees of dependency among each other, we propose in this paper a new search strategy that utilizes dependency between feature pairs to guide the search in the feature space. When compared to other well-known search strategies, the proposed method prevailed.

Adaptive floating search methods in feature selection

A new suboptimal search strategy for feature selection is presented. It represents a more sophisticated version of “classical” floating search algorithms ( Pudil et al., 1994), attempts to remove some of their potential deficiencies and facilitates finding a solution even closer to the optimal one.