Hybrid Feature Selection Method Research Articles

Machine Learning Techniques and Hybrid Feature Selection Methods for Efficient Prediction of Cancer

The high-dimensional genomic data presents significant challenges, and traditional analytical methods often struggle to capture the complex, non-linear relationships within these datasets. This study elaborates into the application of machine learning methods for dimensionality reduction and predictive modeling of binary phenotypes using gene expression data. Various dimensionality reduction techniques are explored, including t-distributed stochastic neighbor embedding (t-SNE), Non-negative matrix factorization (NMF), Principal component analysis (PCA), and manifold learning methods. Additionally, various algorithms such as logistic regression, random forests, support vector machines (SVMs)cand naive Bayes models are evaluated for predicting phenotypes. The study employs rigorous cross-validation, permutation testing, and evaluation metrics like the Matthews Correlation Coefficient (MCC) to assess model performance. The study rigorously assesses current genomics strategies, pinpointing their drawbacks and suggesting areas for future investigation, while delving into the potential of machine learning to overcome these hurdles and offer valuable insights in genomics. Keywords- Gene Expression , Dimensionality Reduction, Biomarkers, Cancer Prediction, Epigenetics, Machine Learning, Feature extraction.

Hybrid feature selection in a machine learning predictive model for perioperative myocardial injury in noncoronary cardiac surgery with cardiopulmonary bypass.

Perioperative myocardial injury (PMI) is associated with increased mobility and mortality after noncoronary cardiac surgery. However, limited studies have developed a predictive model for PMI. Therefore, we used hybrid feature selection (FS) methods to establish a predictive model for PMI in noncoronary cardiac surgery with cardiopulmonary bypass (CPB). This was a single-center retrospective study conducted at the Fuwai Hospital in China. Patients aged 18-70years who underwent elective noncoronary surgery with CPB at our institution from December 2018 to April 2021 were enrolled. The primary outcome was PMI, defined as the postoperative cardiac troponin I (cTnI) levels exceeding 220 times of upper reference limit (URL). Statistical analyses were conducted by Python (Python Software Foundation, version 3.9.7 and integrated development environment Jupyter Notebook 1.1.0) and SPSS software version 26.0 (IBM Corp., Armonk, New York, USA). A total of 1130 patients were eventually eligible for this study. The incidence of PMI was 20.3% (229/1130) in the overall patients, 20.6% (163/791) in the training dataset, and 19.5% (66/339) in the testing dataset. The logistic regression model performed the best AUC of 0.6893 (95 CI%: 0.6371-0.7382) by the traditional selection method, and the random forest model performed the best AUC of 0.6937 (95 CI%: 0.6416-0.7423) by the union of Wrapper and Embedded method, and the CatBoost model performed the best AUC of 0.6828 (95 CI%: 0.6304-0.7320) by the union of Embedded and forward logistic regression technique, and the Naïve Bayes model achieved the best AUC with 0.7254 (95 CI%: 0.6746-0.7723) by forwarding logistic regression method. Moreover, the decision tree, KNeighborsClassifier, and support vector machine models performed the worse AUC in all selection forms. Furthermore, the SHapley Additive exPlanations plot showed that prolonged CPB, aortic clamp time, and preoperative low platelets count were strongly related to the PMI risk. In total, four category feature selection methods were utilized, comprising five individual selection techniques and 15 combined methods. Notably, the combination of logistic regression and embedded methods demonstrated outstanding performance in predicting PMI risk. We also concluded that the machine learning model, including random forest, catboost, and Naive Bayes, were suitable candidates for establishing PMI predictive model. Nevertheless, additional investigation and validation are imperative for substantiating these finding.

A prediction model for water absorption in sublayers based on stacking ensemble learning method

The water absorption profile can reflect the difference in the waterflooding status of each sublayer in the injection well, and predicting the water absorption profile has important guiding significance for formulating and adjusting the waterflood development plan. Currently, numerical simulation and radioisotope logging techniques are commonly used to obtain the water absorption in sublayers. However, the complex and increasingly heterogeneous subsurface formations in China's oilfields have made it impossible for the numerical simulation to reflect the actual reservoir water injection status by splitting the water absorption into sublayers. Although the radioisotope method is more accurate, it is more expensive and has less actual test data in oilfields. In addition, the water absorption prediction model built by traditional machine learning only applies to injection wells with a small amount of water absorption profile data, and its prediction accuracy needs further improvement. In this paper, a new machine learning prediction method is proposed to build a water absorption profile prediction model by stacking ensemble learning algorithm, which achieves the prediction of water absorption in sublayers for injection wells without historical data of water absorption profiles during the development period. In our model, based on the oilfield's static geological data and dynamic production data, the input feature parameters are comprehensively selected by a hybrid feature selection method to analyze the importance of each feature. The robust base-learners are extracted from the sub-model pool by comparing and analyzing the comprehensive prediction performance of each individual model, including ANN, KNN, SVR, RF, XGBoost, LightGBM, etc. The stacking ensemble model was trained using K-fold cross-validation on a randomly divided dataset, which improves the predictive performance of an individual model by combining the advantages of multiple base-learners so that it can accurately reflect the dynamic changes of the water absorption profile. Finally, a case study from the MG study area of the DG oilfield was conducted to verify the model's predictive effect. This case shows that the stacking model proposed in this paper has better performance in predicting the water absorption in sublayers compared with the traditional single model and other ensemble models, and the model has a strong generalization ability, which proves that the method can be used in studying the prediction of the water absorption in sublayers of injection wells.

Review Paper on Cloud Intrusion Detection System

The study proposes an enhanced cloud intrusion detection system (IDS) that tackles security challenges in cloud computing, focusing on data imbalance and feature selection. By integrating SMOTE for data imbalance and a hybrid feature selection method, the system achieves exceptional accuracies exceeding 98% and 99% on two datasets. The use of fewer informative features enhances system efficiency, showcasing its practical applicability and effectiveness in real-world scenarios. Overall, the study contributes significantly to cloud security by offering a holistic approach to IDS enhancement

Empirical Enhancement of Intrusion Detection Systems: A Comprehensive Approach with Genetic Algorithm-based Hyperparameter Tuning and Hybrid Feature Selection

AbstractMachine learning-based IDSs have demonstrated promising outcomes in identifying and mitigating security threats within IoT networks. However, the efficacy of such systems is contingent on various hyperparameters, necessitating optimization to elevate their performance. This paper introduces a comprehensive empirical and quantitative exploration aimed at enhancing intrusion detection systems (IDSs). The study capitalizes on a genetic algorithm-based hyperparameter tuning mechanism and a pioneering hybrid feature selection approach to systematically investigate incremental performance improvements in IDS. Specifically, our work proposes a machine learning-based IDS approach tailored for detecting attacks in IoT environments. To achieve this, we introduce a hybrid feature selection method designed to identify the most salient features for the task. Additionally, we employed the genetic algorithm (GA) to fine-tune hyperparameters of multiple machine learning models, ensuring their accuracy in detecting attacks. We commence by evaluating the default hyperparameters of these models on the CICIDS2017 dataset, followed by rigorous testing of the same algorithms post-optimization through GA. Through a series of experiments, we scrutinize the impact of combining feature selection methods with hyperparameter tuning approaches. The outcomes unequivocally demonstrate the potential of hyperparameter optimization in enhancing the accuracy and efficiency of machine learning-based IDS systems for IoT networks. The empirical nature of our research method provides a meticulous analysis of the efficacy of the proposed techniques through systematic experimentation and quantitative evaluation. Consolidated in a unified manner, the results underscore the step-by-step enhancement of IDS performance, especially in terms of detection time, substantiating the efficacy of our approach in real-world scenarios.

Developing a hybrid feature selection method to detect botnet attacks in IoT devices

The Internet of Things, or IoT, is an important technology applied in various applications such as smart homes and innovative healthcare. Due to its architecture, IoT-based devices suffer from various security challenges, most commonly, botnet attacks. This article aims to develop a hybrid feature selection method to find the most influential features based on three feature selection methods, correlation, generalized normal distribution optimization, and lasso, to detect botnet attacks in IoT devices. The UNSW-NB15 dataset is used to assess the proposed system. Several classification models including decision tree (DT), random forest (RF), k-nearest neighbors (KNN), adaptive boosting (AdaBoost), and bagging are utilized for the classification purpose. The proposed system was evaluated using several performance metrics. The results showed the correlation feature selection method had the most accurate botnet attack detection rate. RF also outperformed other models with a 95.11% detection rate in binary classification and 83.96% in multi-classification. On the other hand, results showed that the proposed hybrid method outperformed the feature selection methods with an increase of about 3% in both classifications. The AdaBoost model achieved an accuracy of 99.28% with binary classification by using 18 features, and the RF model achieved an accuracy of 86.62% with multi-classification by using 22 features. The robustness and efficacy of the proposed approach were demonstrated by comparing the study's results with several other studies that have used the same dataset. The results of the study can be implemented in real applications to detect network interference of a dynamic nature in real-time and assist intrusion detection systems (IDS) in addressing these attacks.

Cloud Intrusion Detection System

Cloud computing is currently reshaping the digital landscape, with a heightened focus on security and privacy concerns for data stored in the cloud. As cyberattacks grow in sophistication and frequency, individuals and organizations alike must prioritize robust intrusion detection systems (IDS). These systems, particularly those utilizing machine learning (ML), excel at identifying network threats but face challenges with large data sizes, leading to decreased performance. Effective feature selection becomes crucial to maintain classification accuracy and prevent information loss. Additionally, addressing imbalanced datasets is vital to mitigate false positives and enhance detection rates. In this study, we propose an enhanced cloud IDS integrating the synthetic minority oversampling technique (SMOTE) for data imbalance and a hybrid feature selection method combining information gain (IG), chi-square (CS), and particle swarm optimization (PSO). Leveraging the random forest (RF) model, our system achieves exceptional accuracies exceeding 98% and 99% on the UNSW-NB15 and Kyoto datasets, respectively. Notably, fewer informative features enhance system efficiency, as evidenced by superior performance compared to existing methodologies

Ensemble feature selection using weighted concatenated voting for text classification

Following the increasing number of high dimensional data, selecting relevant features has always been better handled by filter feature selection techniques due to its improved generalization, faster training time, dimensionality reduction, less prone to overfitting, and improved model performance. However, the most used feature selection methods are unstable; a feature selection method chooses different subsets of characteristics that produce different classification accuracy. Selecting an appropriate hybrid harnesses the local feature relevant to the discriminative power of filter methods for improved text classification, which is lacking in past literature. In this paper, we proposed a novel multi-univariate hybrid feature selection method (MUNIFES) for enhanced discriminative power between the features and the target class. The proposed method utilizes multi-iterative processes to select the best feature sets from each univariate feature selection method. MUNIFES has employed the ensemble of multi-filter discriminative strength of Chi-Square (Chi2), Analysis of Variance (ANOVA), and Infogain methods to select optimal feature subsets. To evaluate the success of the proposed method, several experiments were performed on the 20newsgroup dataset and its variant (17newsgroup) with 10 classifiers (including ensemble, classification and optimization algorithms, and Artificial Neural Network (ANN)), compared with the state-of-the-art feature selection methods. The MUNIFES results indicated a better accuracy classification performance.

Enhancing Intrusion Detection with Explainable AI: A Transparent Approach to Network Security

Abstract An Intrusion Detection System (IDS) is essential to identify cyber-attacks and implement appropriate measures for each risk. The efficiency of the Machine Learning (ML) techniques is compromised in the presence of irrelevant features and class imbalance. In this research, an efficient data pre-processing strategy was proposed to enhance the model’s generalizability. The class dissimilarity is addressed using k-Means SMOTE. After this, we furnish a hybrid feature selection method that combines filters and wrappers. Further, a hyperparameter-tuned Light Gradient Boosting Machine (LGBM) is analyzed by varying the optimal feature subsets. The experiments used the datasets – UNSW-NB15 and CICIDS-2017, yielding an accuracy of 90.71% and 99.98%, respectively. As the transparency and generalizability of the model depend significantly on understanding each component of the prediction, we employed the eXplainable Artificial Intelligence (XAI) method, SHapley Additive exPlanation (SHAP), to improve the comprehension of forecasted results.

Gene selection for high dimensional biological datasets using hybrid island binary artificial bee colony with chaos game optimization

Microarray technology, as applied to the fields of bioinformatics, biotechnology, and bioengineering, has made remarkable progress in both the treatment and prediction of many biological problems. However, this technology presents a critical challenge due to the size of the numerous genes present in the high-dimensional biological datasets associated with an experiment, which leads to a curse of dimensionality on biological data. Such high dimensionality of real biological data sets not only increases memory requirements and training costs, but also reduces the ability of learning algorithms to generalise. Consequently, multiple feature selection (FS) methods have been proposed by researchers to choose the most significant and precise subset of classified genes from gene expression datasets while maintaining high classification accuracy. In this research work, a novel binary method called iBABC-CGO based on the island model of the artificial bee colony algorithm, combined with the chaos game optimization algorithm and SVM classifier, is suggested for FS problems using gene expression data. Due to the binary nature of FS problems, two distinct transfer functions are employed for converting the continuous search space into a binary one, thus improving the efficiency of the exploration and exploitation phases. The suggested strategy is tested on a variety of biological datasets with different scales and compared to popular metaheuristic-based, filter-based, and hybrid FS methods. Experimental results supplemented with the statistical measures, box plots, Wilcoxon tests, Friedman tests, and radar plots demonstrate that compared to prior methods, the proposed iBABC-CGO exhibit competitive performance in terms of classification accuracy, selection of the most relevant subset of genes, data variability, and convergence rate. The suggested method is also proven to identify unique sets of informative, relevant genes successfully with the highest overall average accuracy in 15 tested biological datasets. Additionally, the biological interpretations of the selected genes by the proposed method are also provided in our research work.

A Fast Hybrid Feature Selection Method Based on Dynamic Clustering and Improved Particle Swarm Optimization for High-Dimensional Health Care Data

A Fast Hybrid Feature Selection Method Based on Dynamic Clustering and Improved Particle Swarm Optimization for High-Dimensional Health Care Data

Hybrid feature selection model for accurate wind speed forecasting from numerical weather prediction dataset

The accurate weather prediction in a localized area is obtained as a challenging role and the exact determination of Wind Speed Forecasting (WSF) is suitable for better power generation and production process. However, the Numerical Weather Prediction (NWP) is a significant model to gather information on surface flows and determine the raw data. But it contains issues such as instability, noise, and irregularity that provide a complex situation in the classification process. So Speedy Correlation-based Filtering-Enhanced Adaptive Sparrow Search Wrapper Algorithm (SCEASS-WA) is proposed to assess the problems of redundancy and correlation. Moreover, the Hybrid Feature Selection Method (HFSM) model is employed in the proposed method to control the noise from the extracted features. In SCEASS, the filter model Speedy Correlation Based Filtering (SCBF) based on Symmetrical Uncertainty (SU) is mainly presented to assess the noise. Also, the Wrapper Method (WM) is determined to improve the adaptation of the Sparrow Search Algorithm (SSA) to update the position for better outcomes. Furthermore, the filter methods are varied frequently by obtaining Damping Harmonic Oscillation Theory (DHOT) that discovered the superior feature subset. Also, the high dimensional datasets are employed that are compared with existing methods to show the superiority of the model. The experimentation results revealed that the proposed method attained a better prediction of weather and enhanced the accuracy forecast.

An improved binary particle swarm optimization algorithm for clinical cancer biomarker identification in microarray data

Background and ObjectiveThe limited number of samples and high-dimensional features in microarray data make selecting a small number of features for disease diagnosis a challenging problem. Traditional feature selection methods based on evolutionary algorithms are difficult to search for the optimal set of features in a limited time when dealing with the high-dimensional feature selection problem. New solutions are proposed to solve the above problems. MethodsIn this paper, we propose a hybrid feature selection method (C-IFBPFE) for biomarker identification in microarray data, which combines clustering and improved binary particle swarm optimization while incorporating an embedded feature elimination strategy. Firstly, an adaptive redundant feature judgment method based on correlation clustering is proposed for feature screening to reduce the search space in the subsequent stage. Secondly, we propose an improved flipping probability-based binary particle swarm optimization (IFBPSO), better applicable to the binary particle swarm optimization problem. Finally, we also design a new feature elimination (FE) strategy embedded in the binary particle swarm optimization algorithm. This strategy gradually removes poorer features during iterations to reduce the number of features and improve accuracy. ResultsWe compared C-IFBPFE with other published hybrid feature selection methods on eight public datasets and analyzed the impact of each improvement. The proposed method outperforms other current state-of-the-art feature selection methods in terms of accuracy, number of features, sensitivity, and specificity. The ablation study of this method validates the efficacy of each component, especially the proposed feature elimination strategy significantly improves the performance of the algorithm. ConclusionsThe hybrid feature selection method proposed in this paper helps address the issue of high-dimensional microarray data with few samples. It can select a small subset of features and achieve high classification accuracy on microarray datasets. Additionally, independent validation of the selected features shows that those chosen by C-IFBPFE have strong correlations with disease phenotypes and can identify important biomarkers from data related to biomedical problems.

A novel hybrid feature selection and ensemble-based machine learning approach for botnet detection

In the age of sophisticated cyber threats, botnet detection remains a crucial yet complex security challenge. Existing detection systems are continually outmaneuvered by the relentless advancement of botnet strategies, necessitating a more dynamic and proactive approach. Our research introduces a ground-breaking solution to the persistent botnet problem through a strategic amalgamation of Hybrid Feature Selection methods—Categorical Analysis, Mutual Information, and Principal Component Analysis—and a robust ensemble of machine learning techniques. We uniquely combine these feature selection tools to refine the input space, enhancing the detection capabilities of the ensemble learners. Extra Trees, as the ensemble technique of choice, exhibits exemplary performance, culminating in a near-perfect 99.99% accuracy rate in botnet classification across varied datasets. Our model not only surpasses previous benchmarks but also demonstrates exceptional adaptability to new botnet phenomena, ensuring persistent accuracy in a landscape of evolving threats. Detailed comparative analyses manifest our model's superiority, consistently achieving over 99% True Positive Rates and an unprecedented False Positive Rate close to 0.00%, thereby setting a new precedent for reliability in botnet detection. This research signifies a transformative step in cybersecurity, offering unprecedented precision and resilience against botnet infiltrations, and providing an indispensable blueprint for the development of next-generation security frameworks.

Biomarker detection using corrected degree of domesticity in hybrid social network feature selection for improving classifier performance

BackgroundDimension reduction, especially feature selection, is an important step in improving classification performance for high-dimensional data. Particularly in cancer research, when reducing the number of features, i.e., genes, it is important to select the most informative features/potential biomarkers that could affect the diagnostic accuracy. Therefore, researchers continuously try to explore more efficient ways to reduce the large number of features/genes to a small but informative subset before the classification task. Hybrid methods have been extensively investigated for this purpose, and research to find the optimal approach is ongoing. Social network analysis is used as a part of a hybrid method, although there are several issues that have arisen when using social network tools, such as using a single environment for computing, constructing an adjacency matrix or computing network measures. Therefore, in our study, we apply a hybrid feature selection method consisting of several machine learning algorithms in addition to social network analysis with our proposed network metric, called the corrected degree of domesticity, in a single environment, R, to improve the support vector machine classifier’s performance. In addition, we evaluate and compare the performances of several combinations used in the different steps of the method with a simulation experiment.ResultsThe proposed method improves the classifier’s performance compared to using the whole feature set in all the cases we investigate. Additionally, in terms of the area under the receiver operating characteristic (ROC) curve, our approach improves classification performance compared to several approaches in the literature.ConclusionWhen using the corrected degree of domesticity as a network degree centrality measure, it is important to use our correction to compare nodes/features with no connection outside of their community since it provides a more accurate ranking among the features. Due to the nature of the hybrid method, which includes social network analysis, it is necessary to investigate possible combinations to provide an optimal solution for the microarray data used in the research.

OmicPredict: a framework for omics data prediction using ANOVA-Firefly algorithm for feature selection

High-throughput technologies and machine learning (ML), when applied to a huge pool of medical data such as omics data, result in efficient analysis. Recent research aims to apply and develop ML models to predict a disease well in time using available omics datasets. The present work proposed a framework, ‘OmicPredict’, deploying a hybrid feature selection method and deep neural network (DNN) model to predict multiple diseases using omics data. The hybrid feature selection method is developed using the Analysis of Variance (ANOVA) technique and firefly algorithm. The OmicPredict framework is applied to three case studies, Alzheimer’s disease, Breast cancer, and Coronavirus disease 2019 (COVID-19). In the case study of Alzheimer’s disease, the framework predicts patients using GSE33000 and GSE44770 dataset. In the case study of Breast cancer, the framework predicts human epidermal growth factor receptor 2 (HER2) subtype status using Molecular Taxonomy of Breast Cancer International Consortium (METABRIC) dataset. In the case study of COVID-19, the framework performs patients’ classification using GSE157103 dataset. The experimental results show that DNN model achieved an Area Under Curve (AUC) score of 0.949 for the Alzheimer’s (GSE33000 and GSE44770) dataset. Furthermore, it achieved an AUC score of 0.987 and 0.989 for breast cancer (METABRIC) and COVID-19 (GSE157103) datasets, respectively, outperforming Random Forest, Naïve Bayes models, and the existing research.

An effective hybrid feature selection using entropy weight method for automatic sleep staging

Objective. Sleep staging is the basis for sleep quality assessment and diagnosis of sleep-related disorders. In response to the inadequacy of traditional manual judgement of sleep stages, using machine learning techniques for automatic sleep staging has become a hot topic. To improve the performance of sleep staging, numerous studies have extracted a large number of sleep-related characteristics. However, there are redundant and irrelevant features in the high-dimensional features that reduce the classification accuracy. To address this issue, an effective hybrid feature selection method based on the entropy weight method is proposed in this paper for automatic sleep staging. Approach. Firstly, we preprocess the four modal polysomnography (PSG) signals, including electroencephalogram (EEG), electrooculogram (EOG), electrocardiogram (ECG) and electromyogram (EMG). Secondly, the time domain, frequency domain and nonlinear features are extracted from the preprocessed signals, with a total of 185 features. Then, in order to acquire characteristics of the multi-modal signals that are highly correlated with the sleep stages, the proposed hybrid feature selection method is applied to choose effective features. This method is divided into two stages. In stage I, the entropy weight method is employed to combine two filter methods to build a subset of features. This stage evaluates features based on information theory and distance metrics, which can quickly obtain a subset of features and retain the relevant features. In stage II, Sequential Forward Selection is used to evaluate the subset of features and eliminate redundant features. Further more, to achieve better performance of classification, an ensemble model based on support vector machine, K-nearest neighbor, random forest and multilayer perceptron is finally constructed for classifying sleep stages. Main results. The experiment using the Cyclic Alternating Pattern (CAP) sleep database is performed to assess the performance of the method proposed in this paper. The proposed hybrid feature selection method chooses only 30 features highly correlated to sleep stages. The accuracy, F1 score and Kappa coefficient of 6 class sleep staging reach 88.86%, 83.15% and 0.8531%, respectively. Significance. Experimental results show the effectiveness of the proposed method compared to the existing state-of-the-art studies. It greatly reduces the number of features required while achieving outstanding auto-sleep staging results.

Fast Attack Detection Method for Imbalanced Data in Industrial Cyber-Physical Systems

Abstract Integrating industrial cyber-physical systems (ICPSs) with modern information technologies (5G, artificial intelligence, and big data analytics) has led to the development of industrial intelligence. Still, it has increased the vulnerability of such systems regarding cybersecurity. Traditional network intrusion detection methods for ICPSs are limited in identifying minority attack categories and suffer from high time complexity. To address these issues, this paper proposes a network intrusion detection scheme, which includes an information-theoretic hybrid feature selection method to reduce data dimensionality and the ALLKNN-LightGBM intrusion detection framework. Experimental results on three industrial datasets demonstrate that the proposed method outperforms four mainstream machine learning methods and other advanced intrusion detection techniques regarding accuracy, F-score, and run time complexity.

Breast Cancer Prediction Based on Differential Privacy and Logistic Regression Optimization Model

In order to improve the classification effect of the logistic regression (LR) model for breast cancer prediction, a new hybrid feature selection method is proposed to process the data, using the Pearson correlation test and the iterative random forest algorithm based on out-of-bag estimation (RF-OOB) to screen the optimal 17 features as inputs to the model. Secondly, the LR is optimized using the batch gradient descent (BGD-LR) algorithm to train the loss function of the model to minimize the loss. In order to protect the privacy of breast cancer patients, a differential privacy protection technology is added to the BGD-LR model, and an LR optimization model based on differential privacy with batch gradient descent (BDP-LR) is constructed. Finally, experiments are carried out on the Wisconsin Diagnostic Breast Cancer (WDBC) dataset. Meanwhile, accuracy, precision, recall, and F1-score are selected as the four main evaluation indicators. Moreover, the hyperparameters of each model are determined by the grid search method and the cross-validation method. The experimental results show that after hybrid feature selection, the optimal results of the four main evaluation indicators of the BGD-LR model are 0.9912, 1, 0.9886, and 0.9943, in which the accuracy, recall, and F1-scores are increased by 2.63%, 3.41%, and 1.76%, respectively. For the BDP-LR model, when the privacy budget ε is taken as 0.8, the classification performance and privacy protection effect of the model reach an effective balance. At the same time, the four main evaluation indicators of the model are 0.9721, 0.9975, 0.9664, and 0.9816, which are improved by 1.58%, 0.26%, 1.81%, and 1.07%, respectively. Comparative analysis shows that the models of BGD-LR and BDP-LR constructed in this paper perform better than other classification models.

Ensemble classifiers with hybrid feature selection approach for diagnosis of coronary artery disease

Coronary artery disease (CAD) is a common type of cardiovascular disease with a high mortality rate worldwide. As symptoms may not be recognized until, after the cardiac attack, early diagnosis and treatment are critical to lowering mortality. The proposed study focuses on the creation of an intelligent ensemble system for the accurate detection of CAD. This paper presents the hybrid feature selection method based on Lasso, random forest-based boruta, and recursive feature elimination methods. The significance of a feature is determined by the score each approach provides. Machine learning techniques such as random forest, support vector machine, K-nearest neighbor, logistic regression, decision tree, and Naive Bayes are developed as base classifiers. Then, ensemble techniques like bagging and boosting models are created using base classifiers. The Z-Alizadeh Sani dataset was used to build and test the model. The bagged random forest model achieved 97.6% accuracy and 100% recall. The CatBoost model achieved 97.7% accuracy and 99.0% recall. Compared to traditional classifiers, the ensemble models achieved higher accuracy and can be used to assist clinicians in diagnosing coronary artery disease