Machine Learning Feature Selection Research Articles

Multimodal machine learning-based technical success prediction in patients undergoing transcatheter aortic valve implantation

Abstract Background Procedural success is fundamental to achieving optimal clinical outcome, and prediction of procedural success and failure is of key importance in the selection of the appropriate treatment strategy in patients with severe aortic stenosis requiring aortic valve replacement. Purpose We aimed to develop a prediction model for procedural success in patients undergoing transcatheter aortic valve implantation (TAVI) using multimodal information and employing machine learning algorithms. Method In a prospective TAVI registry, patient baseline demographics, clinical history, medication, laboratory, procedural characteristics, electrocardiogram, angiography, transthoracic and transesophagel echocardiography, and computed tomography information were collected prospectively and extracted as numeric data. Data preprocessing included the imputation of missing values and the elimination of features with high collinearity. We incorporated several feature selection (FS) and classification algorithms for model development. Datasets for each target were randomly split into train/validation sets (70/10%) and hold-out test sets (20%). All machine learning processes, including preprocessing, feature selection, parameter, and hyperparameter optimization, were carried out on training and validation sets in 10-fold cross-validation. Different metrics, including area under the curve (AUC) in receiver operating characteristic curve analyses, sensitivity, and specificity, were calculated for model evaluation on the holdout test set. Procedural success was defined as technical success in the Valve Academic Research Consortium-3 (VARC-3). Technical failure differentiated between vascular and cardiac complications. Results In a total of 2,576 patients undergoing TAVI with a contemporary device between March 2012 and June 2023, 242 (9.4%) had technical failure: 70 (2.7%) had cardiac and 178 (6.9%) had vascular technical failure. The sensitivity and specificity for overall technical failure were 0.71 and 0.77, respectively, with AUC of 0.76 (95% CI: 0.69-0.84) achieved by the recursive feature elimination and AdaBoost algorithm. For cardiac technical failure, the ANOVA FS and Random Forest (RF) model yielded sensitivity and specificity of 0.71 and 0.77, with an AUC of 0.76 (95% CI: 0.65 to 0.87). Vascular technical failure showed AUC of 0.76 (95% CI: 0.69 to 0.84), with sensitivity and specificity of 0.72 and 0.73, respectively, with achieved using the ANOVA FS and RF model. Conclusion The initial results of the novel prediction model for technical success are promising. Further improvements and external validation are warranted to support the use of the proposed model in clinical practice.Central Illustration

Leveraging ML for profiling lipidomic alterations in breast cancer tissues: a methodological perspective

In this study, a comprehensive methodology combining machine learning and statistical analysis was employed to investigate alterations in the metabolite profiles, including lipids, of breast cancer tissues and their subtypes. By integrating biological and machine learning feature selection techniques, along with univariate and multivariate analyses, a notable lipid signature was identified in breast cancer tissues. The results revealed elevated levels of saturated and monounsaturated phospholipids in breast cancer tissues, consistent with external validation findings. Additionally, lipidomics analysis in both the original and validation datasets indicated lower levels of most triacylglycerols compared to non-cancerous tissues, suggesting potential alterations in lipid storage and metabolism within cancer cells. Analysis of cancer subtypes revealed that levels of PC 30:0 were relatively reduced in HER2(−) samples that were ER(+) and PR(+) compared to those that were ER(−) and PR(−). Conversely, HER2(+) tumors, which were ER(−) and PR(−), exhibited increased concentrations of PC 30:0. This increase could potentially be linked to the role of Stearoyl-CoA-Desaturase 1 in breast cancer. Comprehensive metabolomic analyses of breast cancer can offer crucial insights into cancer development, aiding in early detection and treatment evaluation of this devastating disease.

Early monitoring of drought stress in safflower (Carthamus tinctorius L.) using hyperspectral imaging: A comparison of machine learning tools and feature selection approaches

Early detection of drought stress is essential for preventing permanent plant damage and minimizing yield loss. This study utilized hyperspectral imaging at the leaf level to visualize drought stress in safflower plants (Carthamus tinctorius L.). Three safflower genotypes, Palenus, A82, and IL-111, were cultivated under three irrigation levels. Stress conditions were simulated by depleting 50%, 70%, and 90% of soil water content, representing unstressed (US), mild stress (MS), and severe stress (SS) conditions, respectively. Hyperspectral images of leaf samples were captured before any visible signs of water scarcity emerged. Classification analysis was performed using the full mean spectral data with partial least squares discriminant analysis, soft independent modeling of class analogy (SIMCA), support vector machines, and artificial neural network (ANN) classifiers. Feature selection methods were applied to extract the most informative wavebands, and ANN was used to build predictive models. Spatial analysis involved pixel-wise classification using both unsupervised (k-means clustering) and supervised (best classifiers) approaches. ANN outperformed other classifiers using the entire spectral data, effectively distinguishing US, MS, and SS classes in the Palenus, A82, and IL-111 genotypes, achieving F1-scores of 92.22%, 96.01%, and 96.47%, respectively. Among the feature selection methods, SIMCA-based features excelled in monitoring stress conditions in the Palenus and A82 genotypes. In supervised spatial analysis, ANN models clearly depicted the progression of stress in leaves across different genotypes. This study demonstrates the potential of hyperspectral imaging to differentiate various levels of drought stress in safflower, an important oilseed crop.

One-dimensional N-layer thermal modelling for effective machine learning training data generation for nondestructive testing of composite parts with infrared thermography

The analytical solution of the one-dimensional N-layer thermal model (NLM) was successfully employed to generate training data for a machine learning (ML) based procedure for the nondestructive inspection of carbon fiber reinforced composite parts with infrared thermography (IRT). The main objective was to identify a reliable correlation between the experimental data of a pulsed IRT experiment and the NLM prediction, thereby enabling the use of simulated data for ML training. This paper focuses on the initial stages of this process, in more detail on the analytical modelling and experimental data preprocessing, such as normalization, correction of experimental shortcomings and feature selection for machine learning. For pulse phase thermography (PPT) simulated and experimentally derived phase data was compared directly in the frequency domain. Therefore, the features for training and validation of ML were defined from those phase spectra in frequency domain. The suitability of these features for automated and reliable defect depth and/or defect material detection was investigated in both simulated and measured IRT test data. As a basis for feature selection, we used referenced and normalized phase-frequency curves as a function of defect depth. A correlation was identified between the results of the experimental and the simulated feature sets, both qualitatively and quantitatively. To demonstrate the practical applicability of this method, two different, generic ML techniques, multilayer perceptron and random forest regression, were tested as examples. The investigation was performed on plates made of multidirectional carbon fiber reinforced polymer (CFRP) with artificial defects made from three different materials.

Improving Alzheimer's Disease Prediction with Different Machine Learning Approaches and Feature Selection Techniques.

Machine learning (ML) has increasingly been utilized in healthcare to facilitate disease diagnosis and prediction. This study focuses on predicting Alzheimer's disease (AD) through the development and comparison of ML models using Support Vector Machine (SVM), Random Forest (RF), and Logistic Regression (LR) algorithms. Additionally, feature selection techniques including Minimum Redundancy Maximum Relevance (mRMR) and Mutual Information (MI) were employed to enhance the model performance. The research methodology involved training and testing these models on the OASIS-2 dataset, evaluating their predictive accuracies. Notably, LR combined with mRMR achieved the highest accuracy of 99.08% in predicting AD. These findings underscore the efficacy of ML algorithms in AD prediction and highlight the utility of the feature selection methods in improving the model performance. This study contributes to the ongoing efforts to leverage ML for more accurate disease prognosis and underscores the potential of these techniques in advancing clinical decision-making.

Enhancing brain cancer type prediction through machine learning algorithms and feature selection techniques

Abstract The manual identification of brain cancer types is often fraught with inaccuracies, leading to potential delays in diagnosis and treatment planning. This study presents a novel approach to predict brain cancer types using advanced machine learning (ML) algorithms integrated with sophisticated feature selection techniques. A multi-class classification framework was developed and evaluated, incorporating six ML models: Bernoulli Naive Bayes, K-nearest neighbors classifier, decision tree classifier, Gaussian process classifier (GPC), passive aggressive classifier, and perceptron. To enhance model performance, feature selection methods including the Gini index, mutual information, and principal component analysis (PCA) were employed. A comprehensive case study was conducted to assess the predictive accuracy of these models. The GPC, when trained and validated on features derived via PCA, outperformed other models in terms of predictive accuracy and generalization. Specifically, the dimensions identified by PCA (d1, d2, d3, and d4) were most effective in distinguishing between different brain cancer types. This methodology resulted in a significant improvement across various performance metrics. Compared to the baseline GPC model using all original features, the PCA-enhanced GPC achieved remarkable increases in Accuracy, Precision, Recall, and F1 Score by 294.31%, 22.14%, 294.31%, and 878.18%, respectively. These findings underscore the potential of combining ML algorithms with targeted feature selection techniques to advance the accuracy of brain cancer type prediction, offering substantial benefits for clinical decision-making and patient outcomes.

The new machine learning feature selection method used in fertilizer recommendation

The new machine learning feature selection method used in fertilizer recommendation

Enhancing deep learning-based slope stability classification using a novel metaheuristic optimization algorithm for feature selection

The evaluation of slope stability is of crucial importance in geotechnical engineering and has significant implications for infrastructure safety, natural hazard mitigation, and environmental protection. This study aimed to identify the most influential factors affecting slope stability and evaluate the performance of various machine learning models for classifying slope stability. Through correlation analysis and feature importance evaluation using a random forest regressor, cohesion, unit weight, slope height, and friction angle were identified as the most critical parameters influencing slope stability. This research assessed the effectiveness of machine learning techniques combined with modern feature selection algorithms and conventional feature analysis methods. The performance of deep learning models, including recurrent neural networks (RNNs), long short-term memory (LSTM) networks, and generative adversarial networks (GANs), in slope stability classification was evaluated. The GAN model demonstrated superior performance, achieving the highest overall accuracy of 0.913 and the highest area under the ROC curve (AUC) of 0.9285. Integration of the binary bGGO technique for feature selection with the GAN model led to significant improvements in classification performance, with the bGGO-GAN model showing enhanced sensitivity, positive predictive value, negative predictive value, and F1 score compared to the classical GAN model. The bGGO-GAN model achieved 95% accuracy on a substantial dataset of 627 samples, demonstrating competitive performance against other models in the literature while offering strong generalizability. This study highlights the potential of advanced machine learning techniques and feature selection methods for improving slope stability classification and provides valuable insights for geotechnical engineering applications.

Empirical evaluation of feature selection methods for machine learning based intrusion detection in IoT scenarios

This paper delves into the critical need for enhanced security measures within the Internet of Things (IoT) landscape due to inherent vulnerabilities in IoT devices, rendering them susceptible to various forms of cyber-attacks. The study emphasizes the importance of Intrusion Detection Systems (IDS) for continuous threat monitoring. The objective of this study was to conduct a comprehensive evaluation of feature selection (FS) methods using various machine learning (ML) techniques for classifying traffic flows within datasets containing intrusions in IoT environments. An extensive benchmark analysis of ML techniques and FS methods was performed, assessing feature selection under different approaches including Filter Feature Ranking (FFR), Filter-Feature Subset Selection (FSS), and Wrapper-based Feature Selection (WFS). FS becomes pivotal in handling vast IoT data by reducing irrelevant attributes, addressing the curse of dimensionality, enhancing model interpretability, and optimizing resources in devices with limited capacity. Key findings indicate the outperformance for traffic flows classification of certain tree-based algorithms, such as J48 or PART, against other machine learning techniques (naive Bayes, multi-layer perceptron, logistic, adaptive boosting or k-Nearest Neighbors), showcasing a good balance between performance and execution time. FS methods' advantages and drawbacks are discussed, highlighting the main differences in results obtained among different FS approaches. Filter-feature Subset Selection (FSS) approaches such as CFS could be more suitable than Filter Feature Ranking (FFR), which may select correlated attributes, or than Wrapper-based Feature Selection (WFS) methods, which may tailor attribute subsets for specific ML techniques and have lengthy execution times. In any case, reducing attributes via FS has allowed optimization of classification without compromising accuracy. In this study, F1 score classification results above 0.99, along with a reduction of over 60% in the number of attributes, have been achieved in most experiments conducted across four datasets, both in binary and multiclass modes. This work emphasizes the importance of a balanced attribute selection process, taking into account threat detection capabilities and computational complexity.

The credibility assessment of Twitter/X users based organization objectives by heterogeneous resources in big data life cycle

Social network data, such as Twitter/X, is of Big Social Data type. Big social data describes people's social behaviors and interactions. They have high business value for decision-making in organizations. However, because of the anonymous nature of social network users, their credibility is ambiguous. Credibility expresses the accuracy and value of big social data. Despite extensive research on the credibility of big social data, most methods have not paid sufficient attention to the important dimensions of their assessment, including user expertise based on topic, selecting social network features, and labeling them. Furthermore, these methods cannot manage the time, high volume, and speed of big social data. To address these issues, this paper presents a novel model for assessing the credibility of Twitter/X users by integrating Twitter/X with Google Scholar. The model automatically defines users' credibility labels using Google Scholar. Machine learning feature selection methods also select features that affect the credibility of Twitter/X users based on the topic. This study uses Google Scholar and the BerTopic algorithm for effective topic modeling on Twitter/X. The model considers unrelated data management, dynamic user credibility, and organizing activities based on the Big Data lifecycle. Finally, using Linear Regression, Support Vector Regression, K-Nearest Neighbor, Random Forest, Classification and Regression Trees algorithms, the model predicts the credibility of Twitter/X users and proves that it performed better than similar models through Classification and Regression Trees. In addition, the model is generalizable for all organizational purposes due to the integration of heterogeneous resources and feature selection methods.

Optimized Intrusion Detection for IoMT Networks with Tree-Based Machine Learning and Filter-Based Feature Selection.

The Internet of Medical Things (IoMTs) is a network of connected medical equipment such as pacemakers, prosthetics, and smartwatches. Utilizing the IoMT-based system, a huge amount of data is generated, offering experts a valuable resource for tasks such as prediction, real-time monitoring, and diagnosis. To do so, the patient's health data must be transferred to database storage for processing because of the limitations of the storage and computation capabilities of IoMT devices. Consequently, concerns regarding security and privacy can arise due to the limited control over the transmitted information and reliance on wireless transmission, which leaves the network vulnerable to several kinds of attacks. Motivated by this, in this study, we aim to build and improve an efficient intrusion detection system (IDS) for IoMT networks. The proposed IDS leverages tree-based machine learning classifiers combined with filter-based feature selection techniques to enhance detection accuracy and efficiency. The proposed model is used for monitoring and identifying unauthorized or malicious activities within medical devices and networks. To optimize performance and minimize computation costs, we utilize Mutual Information (MI) and XGBoost as filter-based feature selection methods. Then, to reduce the number of the chosen features selected, we apply a mathematical set (intersection) to extract the common features. The proposed method can detect intruders while data are being transferred, allowing for the accurate and efficient analysis of healthcare data at the network's edge. The system's performance is assessed using the CICIDS2017 dataset. We evaluate the proposed model in terms of accuracy, F1 score, recall, precision, true positive rate, and false positive rate. The proposed model achieves 98.79% accuracy and a low false alarm rate 0.007 FAR on the CICIDS2017 dataset according to the experimental results. While this study focuses on binary classification for intrusion detection, we are planning to build a multi-classification approach for future work which will be able to not only detect the attacks but also categorize them. Additionally, we will consider using our proposed feature selection technique for different ML classifiers and evaluate the model's performance empirically in real-world IoMT scenarios.

Performance Analysis of Deep Learning based Signal Constellation Identification Algorithms for Underwater Acoustic Communications

This research delves into the evaluation of Deep learning signal constellation identification (DL-SCI) algorithms in underwater acoustic communications using Orthogonal Frequency Division Multiplexing (OFDM). It distinctly examines at how effective the recurrent neural networks (RNNs), particularly, Gated Recurrent Unit (GRU) and Long Short-Term Memory (LSTM) algorithms in predicting the signal constellation when applied to different underwater acoustic channels characteristics. Unlike manual feature selection in machine learning (ML), in this paper, DL-SCI exploits the labelled OFDM signals at the transmitter to detect and decode them at the receiver. In order to measure their effectiveness performance metrics, Bit Error Rate (BER) and parameters derived from the confusion matrix such as accuracy and precision are used. The study highlights the importance of utilizing zero cyclic prefix techniques which can exploit the inherent bandwidth limitation effectively. Furthermore, when examining complexity, it is observed that both GRU and LSTM algorithms require less floating-point operations (FLOPS) compared to traditional methods such as Minimum Mean Square Error (MMSE) and Least Squares (LS). Interestingly GRU shows performance in terms of complexity when compared to LSTM. Moreover, GRU outperforms LSTM by achieving a 4 dB improvement for long subcarriers. These results emphasize the effectiveness of learning techniques in enhancing performance and efficiency in acoustic communications.

From Data to Decisions: Predictive Machine Learning Models for Customer Retention in Banking

This research proposes machine learning—a subfield of artificial intelligence-based approaches to forecast bank client attrition. The study promotes investigating the possibility of churn through the examination of consumer behaviour. In this paper, a method to predict customer churn in a bank using machine learning techniques, which is a branch of artificial intelligence, is proposed. The research encourages the exploration of the likelihood of churn by analysing customer behaviour. The number of service providers is increasing very rapidly in every business these days. As a result, customer churn and engagement have become one of the top issues for most of the banks. In this work, the Random Forest, SVC, XGB, LGBM, and Logistic Regression classifiers are employed. A few feature selection strategies have also been applied to validate system performance and determine which features are more relevant. The test was conducted using the churn modelling dataset from IEEE dataset. The results are compared to find an appropriate model with increased precision and predictability. As a result, when utilised after oversampling, the Random Forest model outperforms other models in terms of accuracy. The Random Forest model showed a significant improvement in accuracy compared to the other classifiers, making it the preferred choice for predicting customer churn in the banking sector. The feature selection strategies helped identify the most relevant features that contribute to accurate predictions. Overall, this study highlights the importance of implementing advanced machine learning techniques and feature selection methods to enhance customer retention strategies in the banking industry.

Cardiovascular Disease Risk Stratification Using Hybrid Deep Learning Paradigm: First of Its Kind on Canadian Trial Data.

The risk of cardiovascular disease (CVD) has traditionally been predicted via the assessment of carotid plaques. In the proposed study, AtheroEdge™ 3.0HDL (AtheroPoint™, Roseville, CA, USA) was designed to demonstrate how well the features obtained from carotid plaques determine the risk of CVD. We hypothesize that hybrid deep learning (HDL) will outperform unidirectional deep learning, bidirectional deep learning, and machine learning (ML) paradigms. 500 people who had undergone targeted carotid B-mode ultrasonography and coronary angiography were included in the proposed study. ML feature selection was carried out using three different methods, namely principal component analysis (PCA) pooling, the chi-square test (CST), and the random forest regression (RFR) test. The unidirectional and bidirectional deep learning models were trained, and then six types of novel HDL-based models were designed for CVD risk stratification. The AtheroEdge™ 3.0HDL was scientifically validated using seen and unseen datasets while the reliability and statistical tests were conducted using CST along with p-value significance. The performance of AtheroEdge™ 3.0HDL was evaluated by measuring the p-value and area-under-the-curve for both seen and unseen data. The HDL system showed an improvement of 30.20% (0.954 vs. 0.702) over the ML system using the seen datasets. The ML feature extraction analysis showed 70% of common features among all three methods. The generalization of AtheroEdge™ 3.0HDL showed less than 1% (p-value < 0.001) difference between seen and unseen data, complying with regulatory standards. The hypothesis for AtheroEdge™ 3.0HDL was scientifically validated, and the model was tested for reliability and stability and is further adaptable clinically.

Accurate Forecasting of Global Horizontal Irradiance in Saudi Arabia: A Comparative Study of Machine Learning Predictive Models and Feature Selection Techniques

Accurate Forecasting of Global Horizontal Irradiance in Saudi Arabia: A Comparative Study of Machine Learning Predictive Models and Feature Selection Techniques

A review on advancements in feature selection and feature extraction for high-dimensional NGS data analysis.

Recent advancements in biomedical technologies and the proliferation of high-dimensional Next Generation Sequencing (NGS) datasets have led to significant growth in the bulk and density of data. The NGS high-dimensional data, characterized by a large number of genomics, transcriptomics, proteomics, and metagenomics features relative to the number of biological samples, presents significant challenges for reducing feature dimensionality. The high dimensionality of NGS data poses significant challenges for data analysis, including increased computational burden, potential overfitting, and difficulty in interpreting results. Feature selection and feature extraction are two pivotal techniques employed to address these challenges by reducing the dimensionality of the data, thereby enhancing model performance, interpretability, and computational efficiency. Feature selection and feature extraction can be categorized into statistical and machine learning methods. The present study conducts a comprehensive and comparative review of various statistical, machine learning, and deep learning-based feature selection and extraction techniques specifically tailored for NGS and microarray data interpretation of humankind. A thorough literature search was performed to gather information on these techniques, focusing on array-based and NGS data analysis. Various techniques, including deep learning architectures, machine learning algorithms, and statistical methods, have been explored for microarray, bulk RNA-Seq, and single-cell, single-cell RNA-Seq (scRNA-Seq) technology-based datasets surveyed here. The study provides an overview of these techniques, highlighting their applications, advantages, and limitations in the context of high-dimensional NGS data. This review provides better insights for readers to apply feature selection and feature extraction techniques to enhance the performance of predictive models, uncover underlying biological patterns, and gain deeper insights into massive and complex NGS and microarray data.

Explainable machine learning-based fractional vegetation cover inversion and performance optimization – A case study of an alpine grassland on the Qinghai-Tibet Plateau

Fractional Vegetation Cover (FVC) serves as a crucial indicator in ecological sustainability and climate change monitoring. While machine learning is the primary method for FVC inversion, there are still certain shortcomings in feature selection, hyperparameter tuning, underlying surface heterogeneity, and explainability. Addressing these challenges, this study leveraged extensive FVC field data from the Qinghai-Tibet Plateau. Initially, a feature selection algorithm combining genetic algorithms and XGBoost was proposed. This algorithm was integrated with the Optuna tuning method, forming the GA-OP combination to optimize feature selection and hyperparameter tuning in machine learning. Furthermore, comparative analyses of various machine learning models for FVC inversion in alpine grassland were conducted, followed by an investigation into the impact of the underlying surface heterogeneity on inversion performance using the NDVI Coefficient of Variation (NDVI-CV). Lastly, the SHAP (Shapley Additive exPlanations) method was employed for both global and local interpretations of the optimal model. The results indicated that: (1) GA-OP combination exhibited favorable performance in terms of computational cost and inversion accuracy, with Optuna demonstrating significant potential in hyperparameter tuning. (2) Stacking model achieved optimal performance in FVC inversion for alpine grassland among the seven models (R2 = 0.867, RMSE = 0.12, RPD = 2.552, BIAS = −0.0005, VAR = 0.014), with the performance ranking as follows: Stacking > CatBoost > XGBoost > LightGBM > RFR > KNN > SVR. (3) NDVI-CV enhanced inversion performance and result reliability by excluding data from highly heterogeneous regions that tended to be either overestimated or underestimated. (4) SHAP revealed the decision-making processes of the Stacking and CatBoost models from both global and local perspectives. This allowed for a deeper exploration of the causality between features and targets. This study developed a high-precision FVC inversion scheme, successfully achieving accurate FVC inversion on the Qinghai-Tibet Plateau. The proposed approach provides valuable references for other ecological parameter inversions.

Enhancing solar power forecasting with machine learning using principal component analysis and diverse statistical indicators

_Predicting solar energy is essential for efficient power system planning and the successful integration of renewable energy sources. This study aims to develop a framework for evaluating various machine learning models and feature selection strategies for solar energy prediction. The research applies six machine learning models, i.e., linear regression (LR), random forest (RF), neural networks (NN), K-nearest neighbor (KNN), gradient boosting (GB), and AdaBoost, to datasets from 2019 to 2021 collected at the Abiod Sid Cheikh solar station in southern Algeria. Various statistical indicators, including R2, RMSE, MAE, and Adj-R, were analyzed to assess model performance. The analysis revealed that R2 values ranged from 0.591 to 0.996 kW/m2, RMSE from 0.510 to 1.78 kW/m2, and MAE from 0.357 to 0.856 kW/m2 across different models. KNN and NN models showed significant errors, while GB and RF models demonstrated strong accuracies (RMS = 0.9). AdaBoost and LR excelled in real-time and short-term predictions, exhibiting an RMS of 0.99. This framework offers a comprehensive evaluation method for selecting the most suitable machine learning models and feature selection strategies for solar energy prediction. The findings can assist energy planners and engineers in choosing the appropriate models for accurate solar energy prediction, thereby enhancing the efficiency of power systems. Improved solar energy prediction can contribute to more reliable integration of renewable energy into power grids, supporting the transition to cleaner energy sources and reducing environmental impacts.

A Multiform Optimization Framework for Multiobjective Feature Selection in Classification

Feature selection in machine learning as a key data processing technique has two conflicting goals: minimizing the classification error rate and minimizing the number of features selected. However, most of the existing multi-objective feature selection methods face the problems of easily falling into local optima and slow convergence by virtue of their problem characteristics, such as partially conflicting objectives and highly discontinuous Pareto fronts. To solve these problems, this article proposes a multiform optimization framework to solve a multi-objective feature selection task together with several auxiliary single-objective feature selection tasks in a multitask environment. The proposed framework uses the problem-solving experience of single-objective tasks to assist the multi-objective feature selection task in exploring more promising regions and accelerating the convergence speed. Specifically, a knowledge transfer strategy based on the search experience of different tasks is developed to accomplish multiform optimization. In addition, a diversity enhancement mechanism is presented to improve search ability in promising decision space areas by considering historical information about the population. In most cases, the experiment results on 27 datasets demonstrate that the proposed technique can uncover more diversified feature subsets on the Pareto front in less time than existing state-of-the-art methods.

Network Intrusion Detection and Classification System: A Supervised Machine Learning Approach

Abstract: Intrusion detection systems (IDSs) are crucial for computer security, as they identify and counteract malicious activities within computer networks. Anomaly-based IDSs, specifically, use classification models trained on historical data to detect these harmful activities. This paper proposes an enhanced IDS based on 3-level training and testing of machine learning models, feature selection, resampling, and normalization using Decision Tree, Gaussian Naïve Bayes, K-Nearest Neighbours, Logistic Regression, Random Forest, and Support Vector Machine. In the first stage, the six models are trained and evaluated using the original datasets after pre-processing. In the second stage, the models are built and tested with a resampled version of the dataset using the Synthetic Minority Oversampling Technique (SMOTE). In the third stage, the models are trained and tested with a dataset that has been both resampled and normalized using the standard scaling method. We employ the feature importance technique using the random forest model to select the essential features from NSL-KDD and UNSW-NB15 datasets. The results of our study surpass previous related research, with the decision tree achieving an accuracy, precision, recall, and F1 score of 99.99% on the UNSW-NB15 dataset. Additionally, the decision tree recorded an accuracy of 99.98%, precision of 99.97%, recall of 99.97%, and F1 score of 99.99% on the NSL-KDD dataset.