Feature Subset Selection Techniques Research Articles

An optimal feature subset selection technique to improve accounting information security for intrusion detection systems

An optimal feature subset selection technique to improve accounting information security for intrusion detection systems

Improved Decision Support System for College Sports Training Based on Id3 Algorithm

A Decision Support System (DSS) is a computer-based tool designed to assist individuals and organizations in making informed decisions. It utilizes data analysis, modeling techniques, and interactive interfaces to provide users with relevant information and insights. DSSs help streamline decision-making processes by synthesizing large amounts of data from various sources, generating forecasts, and evaluating alternative courses of action. They are particularly valuable in complex and uncertain situations where decisions have significant implications. DSSs can be used in a wide range of domains, including business, healthcare, finance, and logistics, to support strategic, tactical, and operational decision-making processes. This paper introduces an Integrated Hybrid Feature Subset Decision Support System (HFS-DSS) designed to optimize decision-making processes in student performance evaluation within educational institutions. The HFS-DSS combines feature subset selection techniques with decision support algorithms to identify key attributes and make accurate predictions regarding student performance. Through experimental evaluation, the system demonstrates its effectiveness in accurately assessing student performance based on selected features, with predictions closely aligning with actual observations. The results underscore the system's reliability and effectiveness in supporting informed decision-making processes, offering valuable insights for educators and administrators to enhance academic outcomes. The adaptability and versatility of the HFS-DSS make it well-suited for various educational contexts, providing potential applications in curriculum planning, student support services, and performance monitoring. The present paper highlights the significant contributions of the Integrated HFS-DSS towards enhancing decision-making processes and improving academic outcomes in educational institutions.

Enhancing Adaboost performance in the presence of class-label noise: A comparative study on EEG-based classification of schizophrenic patients and benchmark datasets

The performance of Adaboost is highly sensitive to noisy and outlier samples. This is therefore the weights of these samples are exponentially increased in successive rounds. In this paper, three novel schemes are proposed to hunt the corrupted samples and eliminate them through the training process. The methods are: I) a hybrid method based on K-means clustering and K-nearest neighbor, II) a two-layer Adaboost, and III) soft margin support vector machines. All of these solutions are compared to the standard Adaboost on thirteen Gunnar Raetsch’s datasets under three levels of class-label noise. To test the proposed method on a real application, electroencephalography (EEG) signals of 20 schizophrenic patients and 20 age-matched control subjects, are recorded via 20 channels in the idle state. Several features including autoregressive coefficients, band power and fractal dimension are extracted from EEG signals of all participants. Sequential feature subset selection technique is adopted to select the discriminative EEG features. Experimental results imply that exploiting the proposed hunting techniques enhance the Adaboost performance as well as alleviating its robustness against unconfident and noisy samples over Raetsch benchmark and EEG features of the two groups.

Hybrid ML-MDKL feature subset selection and classification technique accompanied with rat swarm optimizer to classify the multidimensional breast cancer mammogram image

Breast cancer is a pathological condition characterized by the abnormal proliferation of cells inside the breast tissue. The biggest challenge at the moment is determining the most optimal subset from large datasets. Numerous methodologies have been presented for the purpose of selecting the most suitable subset from datasets with high dimensions. However, the outcomes have proven to be insufficient in effectively addressing extensive quantities of multidimensional datasets. This publication presents a proposal for an efficient approach of selecting numerous feature subsets and performing classification on Multidimensional Datasets (MDD). In this study, the Moment Invariant Wavelet Feature Extraction (MI-WFE) approach is employed for the purpose of feature extraction. The New Adaptive Hybrid Levy Flight based Cuckoo Search Optimization Algorithm is employed to identify the most pertinent subset of features. This study presents a novel approach, referred to as H-RS-LVCSO, which aims to enhance the local search capability and optimization speed of the algorithm. This is achieved through the hybridization of the Rat Swarm Optimizer with the levy flight based cuckoo search optimization technique. In this study, it is suggested a hybridized approach for classifying breast cancer, utilizing a Multilayer Multiple Deep Kernel Learning (ML-MDKL) Classifier in conjunction with the standard Support Vector Machine (SVM) Classifier. The simulation of the suggested method is conducted using the MATLAB software. The approach under consideration is evaluated in comparison to two pre-existing methods. The suggested method demonstrates an accuracy improvement of 10.58% compared to existing methods. Furthermore, it outperforms these methods by 17.5% in terms of accuracy. The proposed method exhibits a precision improvement of 6.52% and 14.23% compared to the existing methods.

Feature subset selection for data and feature streams: a review

Real-world problems are commonly characterized by a high feature dimensionality, which hinders the modelling and descriptive analysis of the data. However, some of these data may be irrelevant or redundant for the learning process. Different approaches can be used to reduce this information, improving not only the speed of building models but also their performance and interpretability. In this review, we focus on feature subset selection (FSS) techniques, which select a subset of the original feature set without making any transformation on the attributes. Traditional batch FSS algorithms may not be adequate to efficiently handle large volumes of data, either because memory problems arise or data are received in a sequential manner. Thus, this article aims to survey the state of the art of incremental FSS algorithms, which can perform more efficiently under these circumstances. Different strategies are described, such as incrementally updating feature weights, applying information theory or using rough set-based FSS, as well as multiple supervised and unsupervised learning tasks where the application of FSS is interesting.

Multi-label learning by extended multi-tier stacked ensemble method with label correlated feature subset augmentation

<p>Classification is one of the basic and most important operations that can be used in data science and machine learning applications. Multi-label classification is an extension of the multi-class problem where a set of class labels are associated with a particular instance at a time. In a multiclass problem, a single class label is associated with an instance at a time. However, there are many different stacked ensemble methods that have been proposed and because of the complexity associated with the multi-label problems, there is still a lot of scope for improving the prediction accuracy. In this paper, we are proposing the novel extended multi-tier stacked ensemble (EMSTE) method with label correlation<br />by feature subset selection technique and then augmenting those feature subsets while constructing the intermediate dataset for improving the prediction accuracy in the generalization phase of the stacking. The performance effect of the proposed method has been compared with existing methods and showed that our proposed method outperforms the other methods.</p>

Metaheuristics with Deep Learning Model for Cybersecurity and Android Malware Detection and Classification

Since the development of information systems during the last decade, cybersecurity has become a critical concern for many groups, organizations, and institutions. Malware applications are among the commonly used tools and tactics for perpetrating a cyberattack on Android devices, and it is becoming a challenging task to develop novel ways of identifying them. There are various malware detection models available to strengthen the Android operating system against such attacks. These malware detectors categorize the target applications based on the patterns that exist in the features present in the Android applications. As the analytics data continue to grow, they negatively affect the Android defense mechanisms. Since large numbers of unwanted features create a performance bottleneck for the detection mechanism, feature selection techniques are found to be beneficial. This work presents a Rock Hyrax Swarm Optimization with deep learning-based Android malware detection (RHSODL-AMD) model. The technique presented includes finding the Application Programming Interfaces (API) calls and the most significant permissions, which results in effective discrimination between the good ware and malware applications. Therefore, an RHSO based feature subset selection (RHSO-FS) technique is derived to improve the classification results. In addition, the Adamax optimizer with attention recurrent autoencoder (ARAE) model is employed for Android malware detection. The experimental validation of the RHSODL-AMD technique on the Andro-AutoPsy dataset exhibits its promising performance, with a maximum accuracy of 99.05%.

Hybrid Feature Selection based on BTLBO and RNCA to Diagnose the Breast Cancer

Feature selection is a feasible solution to improve the speed and performance of machine learning models. Optimization algorithms are doing a significant job in searching for optimal variables from feature space. Recent feature selection methods are purely depending on various meta heuristic algorithms for searching a good combination of features without considering the importance of individual features, which makes classification models to suffer from local optima or overfitting problems. In this paper, a novel hybrid feature subset selection technique is introduced based on Regularized Neighborhood Component Analysis (RNCA) and Binary Teaching Learning Based Optimization (BTLBO) algorithms to overcome the above problems. RNCA algorithm assigns weights to the attributes based on their contribution in building the learning models for classification. BTLBO algorithm computes the fitness of individuals with respect to the weights of features and selects the best ones. The results of similar feature selection methods are matched with the proposed hybrid model and proved better performance in terms of classification accuracy, recall and AUC measures over breast cancer datasets.

Performance Analysis of Feature Subset Selection Techniques for Intrusion Detection

An intrusion detection system is one of the main defense lines used to provide security to data, information, and computer networks. The problems of this security system are the increased processing time, high false alarm rate, and low detection rate that occur due to the large amount of data containing various irrelevant and redundant features. Therefore, feature selection can solve this problem by reducing the number of features. Choosing appropriate feature selection methods that can reduce the number of features without a negative effect on the classification accuracy is a major challenge. This challenge motivated us to investigate the application of different wrapper feature selection techniques in intrusion detection. The performance of the selected techniques, such as the genetic algorithm (GA), sequential forward selection (SFS), and sequential backward selection (SBS), were analyzed, addressed, and compared to the existing techniques. The efficiency of the three feature selection techniques with two classification methods, including support vector machine (SVM) and multi perceptron (MLP), was compared. The CICIDS2017, CSE-CIC-IDS218, and NSL-KDD datasets were considered for the experiments. The efficiency of the proposed models was proved in the experimental results, which indicated that it had highest accuracy in the selected datasets.

Deep Learning Approaches for Cyberbullying Detection and Classification on Social Media.

As a result of the ease with which the internet and cell phones can be accessed, online social networks (OSN) and social media have seen a significant increase in popularity in recent years. Security and privacy, on the other hand, are the key concerns in online social networks and other social media platforms. On the other hand, cyberbullying (CB) is a serious problem that needs to be addressed on social media platforms. Known as cyberbullying (CB), it is defined as a repetitive, purposeful, and aggressive reaction performed by individuals through the use of information and communication technology (ICT) platforms such as social media platforms, the internet, and cell phones. It is made up of hate messages that are sent by e-mail, chat rooms, and social media platforms, which are accessed through computers and mobile phones. The detection and categorization of CB using deep learning (DL) models in social networks are, therefore, crucial in order to combat this trend. Feature subset selection with deep learning-based CB detection and categorization (FSSDL-CBDC) is a novel approach for social networks that combines deep learning with feature subset selection. The suggested FSSDL-CBDC technique consists of a number of phases, including preprocessing, feature selection, and classification, among others. Additionally, a binary coyote optimization (BCO)-based feature subset selection (BCO-FSS) technique is employed to select a subset of features that will increase classification performance by using the BCO algorithm. Additionally, the salp swarm algorithm (SSA) is used in conjunction with a deep belief network (DBN), which is known to as the SSA-DBN model, to detect and characterize cyberbullying in social media networks and other online environments. The development of the BCO-FSS and SSA-DBN models for the detection and classification of cyberbullying highlights the originality of the research. A large number of simulations were carried out to illustrate the superior classification performance of the proposed FSSDL-CBDC technique. The SSA-DBN model has exhibited superior accuracy to the other algorithms, with a 99.983 % accuracy rate. Overall, the experimental results revealed that the FSSDL-CBDC technique beats the other strategies in a number of different aspects.

Machine Learning Based Depression, Anxiety, and Stress Predictive Model During COVID-19 Crisis

Corona Virus Disease-2019 (COVID-19) was reported at first in Wuhan city, China by December 2019. World Health Organization (WHO) declared COVID-19 as a pandemic i.e., global health crisis on March 11, 2020. The outbreak of COVID-19 pandemic and subsequent lockdowns to curb the spread, not only affected the economic status of a number of countries, but it also resulted in increased levels of Depression, Anxiety, and Stress (DAS) among people. Therefore, there is a need exists to comprehend the relationship among psycho-social factors in a country that is hypothetically affected by high levels of stress and fear; with tremendously-limiting measures of social distancing and lockdown in force; and with high rates of new cases and mortalities. With this motivation, the current study aims at investigating the DAS levels among college students during COVID-19 lockdown since they are identified as a highly-susceptible population. The current study proposes to develop Intelligent Feature Subset Selection with Machine Learning-based DAS predictive (IFSSML-DAS) model. The presented IFSSML-DAS model involves data preprocessing, Feature Subset Selection (FSS), classification, and parameter tuning. Besides, IFSSML-DAS model uses Group Gray Wolf Optimization based FSS (GGWO-FSS) technique to reduce the curse of dimensionality. In addition, Beetle Swarm Optimization based Least Square Support Vector Machine (BSO-LSSVM) model is also employed for classification in which the weight and bias parameters of the LSSVM model are optimally adjusted using BSO algorithm. The performance of the proposed IFSSML-DAS model was tested using a benchmark DASS-21 dataset and the results were investigated under different measures. The outcome of the study suggests the development of specialized programs to handle DAS among population so as to overcome COVID-19 crisis.

Multi‐view rank‐based random forest: A new algorithm for prediction in eSports

AbstractThe main problem associated with the random forest (RF) algorithm is its application of random feature subset selection technique over a single vector. In this technique, the irrelevant or redundant features in a single‐view data have equal chances with the important features to be selected for training a classifier, leading to misclassification. To overcome this problem, this article proposes a novel algorithm, called multi‐view rank‐based random forest (MVRRF). The proposed algorithm builds a set of decision trees from multi‐view data by using a rank‐based feature selection strategy. The main advantages of our method are that (i) it extends the RF algorithm for multi‐view learning, and (ii) it reduces the chances of irrelevant and redundant feature selection, and thus it usually improves the accuracy, generalizability and robustness of the classification models. The aim of our study is to predict the match result of the game League of Legends in electronic sports (eSports). Thus, the eSports teams can define trustworthy strategies through important features. The proposed method can be successfully used in other fields as well as eSports. The experiments that were conducted on a publicly available eSports dataset show that the proposed MVRRF algorithm (93.32%) outperforms the standard RF algorithm (86.38%) on multi‐view data in terms of accuracy. Furthermore, the experimental results also show that our method achieved higher performance than the methods tested in the state of the art studies on the same dataset.

Risk Factors and Comorbidities Associated to Cardiovascular Disease in Qatar: A Machine Learning Based Case-Control Study

Cardiovascular disease (CVD) is reported to be the leading cause of mortality in the middle eastern countries, including Qatar. But no comprehensive study has been conducted on the Qatar specific CVD risk factors identification. The objective of this case-control study was to develop machine learning (ML) model distinguishing healthy individuals from people having CVD, which could ultimately reveal the list of potential risk factors associated to CVD in Qatar. To the best of our knowledge, this study considered the largest collection of biomedical measurements representing the anthropometric measurements, clinical biomarkers, bioimpedance, spirometry, VICORDER readings, and behavioral factors of the CVD group from Qatar Biobank (QBB). CatBoost model achieved 93% accuracy, thereby outperforming the existing model for the same purpose. Interestingly, combining multimodal datasets into the proposed ML model outperformed the ML model built upon currently known risk factors for CVD, emphasizing the importance of incorporating other clinical biomarkers into consideration for CVD diagnosis plan. The ablation study on the multimodal dataset from QBB revealed that physio-clinical and bioimpedance measurements have the most distinguishing power to classify these two groups irrespective of gender and age of the participants. Multiple feature subset selection techniques confirmed known CVD risk factors (blood pressure, lipid profile, smoking, sedentary life, and diabetes), and identified potential novel risk factors linked to CVD-related comorbidities such as renal disorder (e.g., creatinine, uric acid, homocysteine, albumin), atherosclerosis (intima media thickness), hypercoagulable state (fibrinogen), and liver function (e.g., alkaline phosphate, gamma-glutamyl transferase). Moreover, the inclusion of the proposed novel factors into the ML model provides better performance than the model with traditional known risk factors for CVD. The association of the proposed risk factors and comorbidities are required to be investigated in clinical setup to understand their role in CVD better.

Design and Development of an Efficient Network Intrusion Detection System Using Machine Learning Techniques

Today’s internets are made up of nearly half a million different networks. In any network connection, identifying the attacks by their types is a difficult task as different attacks may have various connections, and their number may vary from a few to hundreds of network connections. To solve this problem, a novel hybrid network IDS called NID‐Shield is proposed in the manuscript that classifies the dataset according to different attack types. Furthermore, the attack names found in attack types are classified individually helping considerably in predicting the vulnerability of individual attacks in various networks. The hybrid NID‐Shield NIDS applies the efficient feature subset selection technique called CAPPER and distinct machine learning methods. The UNSW‐NB15 and NSL‐KDD datasets are utilized for the evaluation of metrics. Machine learning algorithms are applied for training the reduced accurate and highly merit feature subsets obtained from CAPPER and then assessed by the cross‐validation method for the reduced attributes. Various performance metrics show that the hybrid NID‐Shield NIDS applied with the CAPPER approach achieves a good accuracy rate and low FPR on the UNSW‐NB15 and NSL‐KDD datasets and shows good performance results when analyzed with various approaches found in existing literature studies.

Text Classification of Cornell Movie Data using Data Mining with Feature Selection

Text Classification is branch of text mining through which we can analyze the sentiment of the movie data. In this research paper we have applied different preprocessing techniques to reduce the features from cornell movie data set. We have also applied the Correlation-based feature subset selection and chi-square feature selection technique for gathering most valuable words of each category in text mining processes. The new cornell movie data set formed after applying the preprocessing steps and feature selection techniques. We have classified the cornell movie data as positive or negative using various classifiers like Support Vector Machine (SVM), Multilayer Perceptron (MLP), Naive Bayes (NB), Bays Net (BN) and Random Forest (RF) classifier. We have also compared the classification accuracy among classifiers and achieved better accuracy i. e. 87% in case of SVM classifier with reduced number of features. The suggested classifier can be useful in opinion of movie review, analysis of any blog and documents etc.

Comprehensive preference learning and feature validity for designing energy-efficient residential buildings using machine learning paradigms

Having a reliable approximation of heating load (HL) and cooling load (CL) is a substantial task for evaluating the energy performance of buildings (EPB). Also, the appearance of soft computing techniques has made many traditional methods antiquated. Thus, the main effort of this study was to evaluate the capability of several learning methods for appraising the HL and CL of a residential building. To this end, a proper dataset consisting of eight influential factors was provided. To simplify the problem, we executed feature validity by using a correlation-based feature subset selection (CfsSubsetEval) technique. The results of this process showed that wall area, overall height, orientation and glazing area have the most significant impact on the HL and CL simulation. After preparing the suitable dataset, sixteen learning methods namely, elastic net (EN), Gaussian process regression (GPR), least median of squares regression (LMSR), multiple linear regression (MLR), multi-layer perceptron regression (MPR), multi-layer perceptron (MLP), radial basis function regression (RBFR), sequential minimal optimization regression (SMOR), functions XNV, lazy K-star, lazy LWL, rules decision table (RDT), M5Rules, alternating model tree (AMT), directional path consistency (DPC), and Random Forest (RF) were developed in Weka environment to forecast the HL and CL variables. Referring to the results, it was concluded that RF, lazy K-star, RDT and AMT outperform other predictive models. Also, comparing the results with the results of the previous studies showed that the applied feature reduction not only did not disturb the learning process but also has enhanced the performance of models. Also, due to the excellent accuracy of the MLP, a formula was derived from the optimized structure of it to predict the HL and CL variables.

Symmetry Based Feature Selection with Multi layer Perceptron for the prediction of Chronic Disease

Huge amount of Healthcare data are produced every day from the various health care sectors. The accumulated data can be effectively analyzed to identify people's risk from chronic diseases. The process of predicting the presence or absence of the disease and also to diagnosing the various disease using the historical medical data is known as Health Care Analytics. Health care analytics will improve patient care and also the harness practice of medical practitioner. The feature selection is considered as a core aspect of the machine learning which hugely contribute towards the performance of the machine learning model. In this paper symmetry based feature subset selection is proposed to select the optimal features from the Health care data which contribute towards the prediction outcome. The Multilayer perceptron algorithm(MLP) used as a classifier which will predict the outcome by using the features which are selected from the Symmetry-based feature subset selection technique. The chronic disease dataset Diabetes, Cancer, Breast Cancer, and Heart Disease data set accumulated from UCI repository is used to conduct the experiment. The experimental results demonstrate that the proposed hybrid combination of feature selection technique and the multilayer perceptron outperforms in accuracy compare to the existing approaches.

Predictive Models for Classifying the Outcomes of Violence: Case Study for Thailand’s Deep South

Violence is now widely recognized as a public health problem because of its significant consequences on the health and wellness of people and it remains a growing problem in many countries including Thailand. Elucidating the factors related to violence can provide information that can help to prevent violence and decrease the number of injuries. This study explored predictive data mining models which have high interpretability and prediction accuracy in classifying the outcomes of violence. After data preprocessing, a set of 21,424 incidents occurring from 2004 to 2016 were obtained from the Deep South Coordination Centre database. A correlation-based feature subset selection and decision tree technique with embedded feature selection were used for variable selection and four data mining techniques were applied to classify the violent outcomes into physical injury and no physical injury. The findings revealed that regardless of the variable selection method, gun was selected as a risk factor of physical injury. Moreover, a decision tree model with three variables, gun, zone, and solid/sharp weapon outperformed a naive Bayes model in terms of accurate performance and interpretability. Decision tree and artificial neural network models have similar levels of performance in classifying the outcome of violence but in practical terms, a decision tree model is more interpretable than an artificial neural network model.

Developing intelligent classification models for rock burst prediction after recognizing significant predictor variables, Section 1: Literature review and data preprocessing procedure

Rock bursts constitute serious hazards in underground mining and excavating. Up to now, numerous researches in the form of empirical, experimental, analytical, intelligent and numerical methods with their own specific scope, characteristics, strengths and weaknesses, have been conducted for rock burst prediction. The weaknesses and limitations of the mentioned prediction methods, especially the intelligent studies, indicate the need for continuing the researches in this field. In this research, a rock burst database, consisting of 188 datasets, was considered. Each dataset corresponds to a series of predictor variables and one of defined classes for the dependent variable “rock burst intensity”. To design classification models, describing important characteristics of datasets and predicting future trends, a data preprocessing procedure was conducted. The procedure consisted of a statistical analysis strategy, a metaheuristic technique for feature (variable) subset selection and some feature extraction techniques. The statistical analysis led to conclude that by considering the available datasets, some predictor variables have statistically insignificant contributions for rock burst prediction. By contrast, the other predictor variables have considerable ordinal contributions. These statistical inferences were completely in accordance with the results of the feature subset selection technique. Besides, the application of this technique revealed specific combinations of significant predictor variables having the highest priorities for modelling the dependent variable. The application of feature extraction techniques to construct derived components from initial datasets did not lead to representative results. Therefore, a high rank combination of significant predictor variables can be adopted to design and develop new classification models based on the considered datasets.

Correlation-based feature subset selection technique for web spam classification

In past years, different machine learning algorithms and web spam features have been created to recognise the spam. The key part of progression of machine learning (ML) depends on the features being utilised. If we have features which correlate with each other then it is easy for ML to learn and if we have features which are very complex then ML may not be able to learn. It is the most imperative and basic area where the majority of the applications in a machine learning are going on. In this paper, correlation-based feature selection (CFS) technique (with best-first search) is used which selects features that are most efficient. Two datasets (WebSpam-UK2006 and WebSpam-UK2007) and four classifiers (Naive Bayes, J48, random forest and AdaBoost) are used for conducting the experiment. The results have shown significant improvement in AUC (area under receiver operating characteristic curve) for Naive Bayes and J48.