Hybrid Feature Selection Model Research Articles

A hybrid feature selection model for identifying groups of critical elements in aero‐engine assembly

AbstractAero‐engine assembly has a wide range of indicators, and finding critical assembly elements to guide assembly and troubleshooting is essential in improving the reliability and safety of aero‐engine. In order to identify critical elements in aero‐engine assembly components, this study aims to establish a two‐stage hybrid feature selection model, namely, FSBP approach, which integrated filter method and particle swarm algorithm with Bayesian optimization. Specifically, individual filter feature selection methods are compared to select a relatively effective method to reduce the data dimensions and ensure the quality of the initial subset. Then, the particle swarm algorithm combined with Bayesian optimization obtains a subset of features in the second stage that are more suitable for predictive models with more robust classification prediction capability. The algorithm is successfully applied to real aero‐engine assembly and trial test datasets, and the experimental results show that our proposed two‐stage hybrid feature selection model outperforms other benchmark methods.

Hybrid feature selection model for accurate wind speed forecasting from numerical weather prediction dataset

The accurate weather prediction in a localized area is obtained as a challenging role and the exact determination of Wind Speed Forecasting (WSF) is suitable for better power generation and production process. However, the Numerical Weather Prediction (NWP) is a significant model to gather information on surface flows and determine the raw data. But it contains issues such as instability, noise, and irregularity that provide a complex situation in the classification process. So Speedy Correlation-based Filtering-Enhanced Adaptive Sparrow Search Wrapper Algorithm (SCEASS-WA) is proposed to assess the problems of redundancy and correlation. Moreover, the Hybrid Feature Selection Method (HFSM) model is employed in the proposed method to control the noise from the extracted features. In SCEASS, the filter model Speedy Correlation Based Filtering (SCBF) based on Symmetrical Uncertainty (SU) is mainly presented to assess the noise. Also, the Wrapper Method (WM) is determined to improve the adaptation of the Sparrow Search Algorithm (SSA) to update the position for better outcomes. Furthermore, the filter methods are varied frequently by obtaining Damping Harmonic Oscillation Theory (DHOT) that discovered the superior feature subset. Also, the high dimensional datasets are employed that are compared with existing methods to show the superiority of the model. The experimentation results revealed that the proposed method attained a better prediction of weather and enhanced the accuracy forecast.

Automatic Diagnosis of Autism Spectrum Disorder Detection Using a Hybrid Feature Selection Model with Graph Convolution Network

Automatic Diagnosis of Autism Spectrum Disorder Detection Using a Hybrid Feature Selection Model with Graph Convolution Network

A hybrid machine learning feature selection model-HMLFSM to enhance gene classification applied to multiple colon cancers dataset.

Colon cancer is a significant global health problem, and early detection is critical for improving survival rates. Traditional detection methods, such as colonoscopies, can be invasive and uncomfortable for patients. Machine Learning (ML) algorithms have emerged as a promising approach for non-invasive colon cancer classification using genetic data or patient demographics and medical history. One approach is to use ML to analyse genetic data, or patient demographics and medical history, to predict the likelihood of colon cancer. However, due to the challenges imposed by variable gene expression and the high dimensionality of cancer-related datasets, traditional transductive ML applications have limited accuracy and risk overfitting. In this paper, we propose a new hybrid feature selection model called HMLFSM-Hybrid Machine Learning Feature Selection Model to improve colon cancer gene classification. We developed a multifilter hybrid model including a two-phase feature selection approach, combining Information Gain (IG) and Genetic Algorithms (GA), and minimum Redundancy Maximum Relevance (mRMR) coupling with Particle Swarm Optimization (PSO). We critically tested our model on three colon cancer genetic datasets and found that the new framework outperformed other models with significant accuracy improvements (95%, ~97%, and ~94% accuracies for datasets 1, 2, and 3 respectively). The results show that our approach improves the classification accuracy of colon cancer detection by highlighting important and relevant genes, eliminating irrelevant ones, and revealing the genes that have a direct influence on the classification process. For colon cancer gene analysis, and along with our experiments and literature review, we found that selective input feature extraction prior to feature selection is essential for improving predictive performance.

Breast cancer classification using a novel hybrid feature selection approach

Many women around the world die due to breast cancer. If breast cancer is treated in the early phase, mortality rates may significantly be reduced. Quite a number of approaches have been proposed to help in the early detection of breast cancer. A novel hybrid feature selection model is suggested in this study. This novel hybrid model aims to build an efficient feature selection method and successfully classify breast lesions. A combination of relief and binary Harris hawk optimization (BHHO) hybrid model is used for feature selection. Then, k-nearest neighbor (k-NN), support vector machine (SVM), logistic regression (LR) and naive Bayes (NB) methods are preferred for the classification task. The suggested hybrid model is tested by three different breast cancer datasets which are Wisconsin diagnostic breast cancer dataset (WDBC), Wisconsin breast cancer dataset (WBCD) and mammographic breast cancer dataset (MBCD). According to the experimental results, the relief and BHHO hybrid model improves the performance of all classification algorithms in all three datasets. For WDBC, relief-BHO-SVM model shows the highest classification rates with an of accuracy of 98.77%, precision of 97.17%, recall of 99.52%, F1-score of 98.33%, specificity of 99.72% and balanced accuracy of 99.62%. For WBCD, relief-BHO-SVM model achieves of accuracy of 99.28%, precision of 98.76%, recall of 99.17%, F1-score of 98.96%, specificity of 99.56% and balanced accuracy of 99.36%. Relief-BHO-SVM model performs the best with an accuracy of 97.44%, precision of 97.41%, recall of 98.26%, F1-score of 97.84%, specificity of 97.47% and balanced accuracy of 97.86% for MBCD. Furthermore, the relief-BHO-SVM model has achieved better results than other known approaches. Compared with recent studies on breast cancer classification, the suggested hybrid method has achieved quite good results.

An Efficient Allocation for Lung Transplantation Using Ant Colony Optimization

A relationship between lung transplant success and many features of recipients’/donors has long been studied. However, modeling a robust model of a potential impact on organ transplant success has proved challenging. In this study, a hybrid feature selection model was developed based on ant colony optimization (ACO) and k-nearest neighbor (kNN) classifier to investigate the relationship between the most defining features of recipients/donors and lung transplant success using data from the United Network of Organ Sharing (UNOS). The proposed ACO-kNN approach explores the features space to identify the representative attributes and classify patients’ functional status (i.e., quality of life) after lung transplantation. The efficacy of the proposed model was verified using 3,684 records and 118 input features from the UNOS. The developed approach examined the reliability and validity of the lung allocation process. The results are promising regarding accuracy prediction to be 91.3% and low computational time, along with better decision capabilities, emphasizing the potential for automatic classification of the lung and other organs allocation processes. In addition, the proposed model recommends a new perspective on how medical experts and clinicians respond to uncertain and challenging lung allocation strategies. Having such ACO-kNN model, a medical professional can summarize information through the proposed method and make decisions for the upcoming transplants to allocate the donor organ.

A Boosting-Based Hybrid Feature Selection and Multi-Layer Stacked Ensemble Learning Model to Detect Phishing Websites

A Boosting-Based Hybrid Feature Selection and Multi-Layer Stacked Ensemble Learning Model to Detect Phishing Websites

A hybrid machine learning technique for feature optimization in object-based classification of debris-covered glaciers

Object-based features like spectral, topographic, and textural are supportive to determine debris-covered glacier classes. The original feature space includes relevant and irrelevant features. The inclusion of all these features increases the complexity and renders the classifier’s performance. Therefore, feature space optimization is requisite for the classification process. Previous studies have shown a rigorous exercise in manually selecting the best combination of features to define the target class and proven to be a time-consuming task. The present study proposed a hybrid feature selection technique to automate the selection of the best suitable features. This study aimed to reduce the classifier’s complexity and enhance the performance of the classification model. Relief-F and Pearson Correlation filter-based feature selection methods ranked features according to the relevance and filtered out irrelevant or less important features based on the defined condition. Later, the hybrid model selected the common features to get an optimal feature set. The proposed hybrid model was tested on Landsat 8 images of debris-covered glaciers in Central Karakoram Range and validated with present glacier inventories. The results showed that the classification accuracy of the proposed hybrid feature selection model with a Decision Tree classifier is 99.82%, which is better than the classification results obtained using other mapping techniques. In addition, the hybrid feature selection technique has sped up the process of classification by reducing the number of features by 77% without compromising the classification accuracy.

A hybrid feature selection algorithm using simplified swarm optimization for body fat prediction

Background and objectivesObesity is one of the chronic diseases that seriously threaten people's health outcomes globally. Since the prevalence of obesity is increasing among people of all ages, measuring the body fat percentages is vital before treatment. However, the body fat percentage cannot be accurately measured by weighing. While many devices are commonly used to measure the body fat percentage, these devices are expensive and depend on complex instruments. Therefore, more practical and cost-effective solutions are desired to measure body fat accurately. This study presents a hybrid feature selection method based on a VIKOR-based multi-filter ensemble technique (VMFET) and an improved simplified swarm optimization (iSSO) to predict the body fat percentage with low prediction error. MethodsThe study followed a two-phase process. First, VMFET was used to aggregate the statistical outcomesof individual filters to filter the most informative features from the original dataset. Then, the selected features are applied to the next phase. Second, iSSO was tailored with a biased random initialization scheme, effect-based feature pruning scheme, and multiple linear regression as a wrapper method to improve the prediction performance and select the optimal feature subset. Studies resultsExtensive experiments were performed using nine datasets to verify the performance of the proposed method empirically, and the corresponding results were compared with up-to-date studies. ConclusionThe statistical results demonstrated that the proposed method offers a promising and effective tool for predicting body fat. SignificanceThe hybrid feature selection model can enhance prediction accuracy and lower prediction error.

A Deep Learning-Based Framework for Feature Extraction and Classification of Intrusion Detection in Networks

An intrusion detection system, often known as an IDS, is extremely important for preventing attacks on a network, violating network policies, and gaining unauthorized access to a network. The effectiveness of IDS is highly dependent on data preprocessing techniques and classification models used to enhance accuracy and reduce model training and testing time. For the purpose of anomaly identification, researchers have developed several machine learning and deep learning-based algorithms; nonetheless, accurate anomaly detection with low test and train times remains a challenge. Using a hybrid feature selection approach and a deep neural network- (DNN-) based classifier, the authors of this research suggest an enhanced intrusion detection system (IDS). In order to construct a subset of reduced and optimal features that may be used for classification, a hybrid feature selection model that consists of three methods, namely, chi square, ANOVA, and principal component analysis (PCA), is applied. These methods are referred to as “the big three.” On the NSL-KDD dataset, the suggested model receives training and is then evaluated. The proposed method was successful in achieving the following results: a reduction of input data by 40%, an average accuracy of 99.73%, a precision score of 99.75%, an F1 score of 99.72%, and an average training and testing time of 138% and 2.7 seconds, respectively. The findings of the experiments demonstrate that the proposed model is superior to the performance of the other comparison approaches.

Machine Learning based Early Prediction of Type 2 Diabetes: A New Hybrid Feature Selection Approach using Correlation Matrix with Heatmap and SFS

A new hybrid machine learning method for the prediction of type 2 diabetes is introduced and explained in detail. Also, outcomes are compared with similar researches. Early prediction of diabetes is crucial to take necessary measures (i.e. changing eating habits, patient weight control etc.), to defer the emergence of diabetes and to reduce the death rate to some extent and ease medical care professionals’ decision-making in preventing and managing diabetes mellitus. The purpose of this study is the creation of a new hybrid feature selection approach combination of Correlation Matrix with Heatmap and Sequential forward selection (SFS) to reveal the most effective features in the detection of diabetes. A diabetes data set with 520 instances and seven features were studied with the application of the proposed hybrid feature selection approach. The evaluation of the selected optimal features was measured by applying Support Vector Machines(SVM), Random Forest(RF), and Artificial Neural Networks(ANN) classifiers. Five evaluation metrics, namely, Accuracy, F-measure, Precision, Recall, and AUC showed the best performance with ANN (99.1%), F-measure (99.1%), Precision (99.3%), Recall (99.1%), and AUC (99.2%). Our proposed hybrid feature selection model provided a more promising performance with ANN compared to other machine learning algorithms.

Bio-inspired Hybrid Feature Selection Model for Intrusion Detection

Intrusion detection is a serious and complex problem. Undoubtedly due to a large number of attacks around the world, the concept of intrusion detection has become very important. This research proposes a multilayer bio-inspired feature selection model for intrusion detection using an optimized genetic algorithm. Furthermore, the proposed multilayer model consists of two layers (layers 1 and 2). At layer 1, three algorithms are used for the feature selection. The algorithms used are Particle Swarm Optimization (PSO), Grey Wolf Optimization (GWO), and Firefly Optimization Algorithm (FFA). At the end of layer 1, a priority value will be assigned for each feature set. At layer 2 of the proposed model, the Optimized Genetic Algorithm (GA) is used to select one feature set based on the priority value. Modifications are done on standard GA to perform optimization and to fit the proposed model. The Optimized GA is used in the training phase to assign a priority value for each feature set. Also, the priority values are categorized into three categories: high, medium, and low. Besides, the Optimized GA is used in the testing phase to select a feature set based on its priority. The feature set with a high priority will be given a high priority to be selected. At the end of phase 2, an update for feature set priority may occur based on the selected features priority and the calculated F-Measures. The proposed model can learn and modify feature sets priority, which will be reflected in selecting features. For evaluation purposes, two well-known datasets are used in these experiments. The first dataset is UNSW-NB15, the other dataset is the NSL-KDD. Several evaluation criteria are used, such as precision, recall, and F-Measure. The experiments in this research suggest that the proposed model has a powerful and promising mechanism for the intrusion detection system.

A Novel Feature Selection Method With Neighborhood Rough Set and Improved Particle Swarm Optimization

Neighborhood rough set is an excellent mathematical tool to carry out feature selection on both numerical and categorical dataset. However, conventional feature selection algorithms usually use greedy heuristic search strategies, which is easy to trap in the local extreme point. In this study, we propose a hybrid feature selection model that combines neighborhood rough set with an improved particle swarm optimization. The model computes the dependency degree of decision from neighborhood rough set as the objective function to evaluate the selected features, and then takes advantages of PSO’s stochastic search to discover the optimal solution more effectively. In order to improve the global search ability and alleviate the stagnation in local optima, the model modifies PSO part by adopting genetic operators and Levy flight. To evaluate the performance of this model, we implement experiments using twelve benchmark datasets and two classifiers ( <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$k$ </tex-math></inline-formula> NN and SVM). Compared with five representative filter-based approaches, experimental results show that our model can not only guarantee the stronger classification ability but also remove more redundant features in most datasets.

Hybrid feature selection model based on machine learning and knowledge graph

Aiming at the problem that the current feature selection algorithm can not adapt to both supervised learning data and unsupervised learning data, and had poor feature interpretability, this paper proposed a hybrid feature selection model based on machine learning and knowledge graph. By the idea of hybridization, this model used supervised learning algorithms, unsupervised learning algorithms and knowledge graph technology to model from the perspective of data features and text features. Firstly, the data-based feature weights were obtained through the machine learning model, and then the text-based weights were obtained by using the knowledge graph technology, and the weight sets are combined to obtain a feature matrix with good explanatory properties that meets both the data and text features. Finally, the case analysis proves that the method proposed in this paper has good effects and interpretability.

Palm Vein Identification Based on Hybrid Feature Selection Model

Palm Vein Identification (PVI) is a modern biometric security technique used for enhancing security and authentication systems. The key characteristics of palm vein patterns include its uniqueness to each individual, its unforgettability, non-intrusiveness and its ability for disallowing unauthorized persons. However, the extracted features from the palm vein patterns are huge with high redundancy. In this paper, we propose a combined model of two-Dimensional Discrete Wavelet Transform, Principal Component Analysis (PCA), and Particle Swarm Optimization (PSO) (2D-DWTPP) that feeds wrapper model with an optimal subset of features to enhance the prediction accuracy of -palm vein patterns. The 2D-DWT extract features from palm vein images, using the PCA to reduce the redundancy in palm vein features. The system has been trained to select high recognition features based on the wrapper model. The proposed system uses four classifiers as an objective function to determine PVI which include Support Vector Machine (SVM), K Nearest Neighbor (KNN), Decision Tree (DT) and Naive Bayes (NB). The empirical results proved that the proposed model has the best results with SVM. Moreover, our proposed 2D-DWTPP model has been evaluated and the results show remarkable efficiency in comparison with AlexNet and other classifiers without feature selection. Experimentally, the proposed model has better accuracy as reflected by 98.65% whereas AlexNet has 63.5% accuracy and the classifier without feature selection process has 78.79% accuracy.

User Authentication Based on Handwriting Analysis of Pen-Tablet Sensor Data Using Optimal Feature Selection Model

Handwriting analysis is playing an important role in user authentication or online writer identification for more than a decade. It has a significant role in different applications such as e-security, signature biometrics, e-health, gesture analysis, diagnosis system of Parkinson’s disease, Attention-deficit/hyperactivity disorders, analysis of vulnerable people (stressed, elderly, or drugged), prediction of gender, handedness and so on. Classical authentication systems are image-based, text-dependent, and password or fingerprint-based where the former one has the risk of information leakage. Alternatively, image processing and pattern-analysis-based systems are vulnerable to camera attributes, camera frames, light effect, and the quality of the image or pattern. Thus, in this paper, we concentrate on real-time and context-free handwriting data analysis for robust user authentication systems using digital pen-tablet sensor data. Most of the state-of-the-art authentication models show suboptimal performance for improper features. This research proposed a robust and efficient user identification system using an optimal feature selection technique based on the features from the sensor’s signal of pen and tablet devices. The proposed system includes more genuine and accurate numerical data which are used for features extraction model based on both the kinematic and statistical features of individual handwritings. Sensor data of digital pen-tablet devices generate high dimensional feature vectors for user identification. However, all the features do not play equal contribution to identify a user. Hence, to find out the optimal features, we utilized a hybrid feature selection model. Extracted features are then fed to the popular machine learning (ML) algorithms to generate a nonlinear classifier through training and testing phases. The experimental result analysis shows that the proposed model achieves more accurate and satisfactory results which ensure the practicality of our system for user identification with low computational cost.

A hybrid feature selection model based on butterfly optimization algorithm: COVID-19 as a case study.

The need to evolve a novel feature selection (FS) approach was motivated by the persistence necessary for a robust FS system, the time‐consuming exhaustive search in traditional methods, and the favourable swarming manner in various optimization techniques. Most of the datasets have a high dimension in many issues since all features are not crucial to the problem, which reduces the algorithm's accuracy and efficiency. This article presents a hybrid feature selection approach to solve the low precision and tardy convergence of the butterfly optimization algorithm (BOA). The proposed method is dependent on combining the algorithm of BOA and the particle swarm optimization (PSO) as a search methodology using a wrapper framework. BOA is started with a one‐dimensional cubic map in the proposed approach, and a non‐linear parameter control technique is also implemented. To boost the basic BOA for global optimization, PSO algorithm is mixed with the butterfly optimization algorithm (BOAPSO). A 25 dataset evaluates the proposed BOAPSO to determine its efficiency with three metrics: classification precision, the selected features, and the computational time. A COVID‐19 dataset has been used to evaluate the proposed approach. Compared to the previous approaches, the findings show the supremacy of BOAPSO for enhancing performance precision and minimizing the number of chosen features. Concerning the accuracy, the experimental outcomes demonstrate that the proposed model converges rapidly and performs better than with the PSO, BOA, and GWO with improvement percentages: 91.07%, 87.2%, 87.8%, 87.3%, respectively. Moreover, the proposed model's average selected features are 5.7 compared to the PSO, BOA, and GWO, with average features 22.5, 18.05, and 23.1, respectively.

A hybrid ensemble-filter wrapper feature selection approach for medical data classification

Background and objectiveMedical data plays a decisive role in disease diagnosis. The classification accuracy of high-dimensional datasets is often diminished by several redundant and irrelevant features. In this context, feature selection becomes an indispensable process. Feature selection primarily intends to identify a feature subspace which retains the classification accuracy while reducing the high computational cost of learning model as well as eliminating noise. The suitability of an appropriate feature selection approach heavily depends upon the capability of that approach to match the problem framework and to discover the intrinsic patterns within the data. The prime objective of this paper is to develop an ensemble-filter based hybrid feature selection model for disease detection. MethodsIn this paper, a four-step hybrid ensemble feature selection algorithm has been introduced. Firstly, the dataset is partitioned using the cross-validation procedure. Secondly, in the filter step, various filter methods based on weighted scores were ensembled to generate a ranking of features, and thirdly sequential forward selection algorithm is utilized as a wrapper technique to obtain an optimal subset of features. Finally, the resulting optimal subset is processed for subsequent classification tasks. ResultsExperiments have been performed on twenty benchmark medical datasets with different dimensionalities. The performance of the proposed hybrid approach is compared with fourteen state-of-the-art feature selection algorithms on four benchmark classifiers namely, Naïve Bayes, Support Vector Machine with Radial Basis Function, Random Forest and k-Nearest Neighbor. The empirical results clearly demonstrate the superiority of the proposed hybrid methodology over the competing methods with respect to accuracy, sensitivity, specificity, f1-score, area under curve evaluation measures and number of selected features. The statistical analysis of the obtained results shows the outperformance and the competitiveness of the proposed hybrid method with respect to various state-of-the-art algorithms. ConclusionsThis study concludes that the proposed hybrid approach proves to be more effective and reliable feature selection technique in selecting highly discriminative features. The framework can be utilized as a promising tool by both clinicians and researchers in enhancing the classification performance of medical datasets.

A hybrid feature selection model based on improved squirrel search algorithm and rank aggregation using fuzzy techniques for biomedical data classification.

Feature selection has gained its importance due to the voluminous nature of the data. Owing to the computational complexity of wrapper approaches, the poor performance of filtering techniques, and the classifier dependency of embedded approaches, hybrid approaches are more commonly used in feature selection. Hybrid approaches use filtering metrics to reduce the computational complexity of wrapper algorithms and are proved to yield better feature subset. Though filtering metrics select the features based on their significance, most of them are unstable and biased towards the metric used. Moreover, the choice of filtering metrics depends largely on the distribution of data and data types. Biomedical datasets contain features with different distribution and types adding to the complexity in the choice of filtering metric. We address this problem by proposing a stable filtering method based on rank aggregation in hybrid feature selection model with Improved Squirrel search algorithm for biomedical datasets. Our proposed model is compared with other well-known and state-of-the-art methods and the results prove that our model exhibited superior performance in terms of classification accuracy and computational time. The robustness of our proposed model is proved by conducting experiments on nine biomedical datasets and with three different classifiers.

Comparison of machine learning algorithms for chest X-ray image COVID-19 classification

Artificial Intelligence and Machine Learning algorithms were used to identify the coronavirus (COVID-19) from X-ray photos of the chest. The authors propose a model for early coronavirus detection based on image filtering strategies and a hybrid feature selection model in this analysis. Traditional statistical and machine learning methods are used to derive these attributes from CT images. The Confusion Matrix for infected COVID-19 patients and regular patients was obtained using Support Vector Machine and K-Nearest Neighbor to classify the features chosen. The output of the two approaches can be compared. The various techniques’ performance shows that the Support Vector Machine achieves the highest precision of 97% compared to the K-Nearest Neighbor with a precision of 86%.