Ensemble Algorithm Research Articles

An Ensemble Learning Algorithm with Hyperparameter Optimization (ELAHO) Model for Early Detection of Maternal Health Risk (MHR) Level Using Machine Learning

Despite the advancement in the healthcare system, maternal health risk remains high, which is the most challenging aspect nowadays. There is a need to develop an effective model for early detection, monitoring, and prediction of maternal health risk levels during pregnancy. The machine learning intelligence model has proven its effectiveness and robustness in providing accurate and reliable prediction, analysis, and interpretation of medical data, reducing several risk factors for early diagnosis in healthcare. In this research work, we proposed an ensemble learning algorithm with a hyperparameter optimization (ELAHO) model using machine learning algorithms to improve its robustness, effectiveness, and model performance. The proposed method uses a hybrid model of logistic regression and support vector machines (LG-SVM) to predict maternal health risk levels during pregnancy. The method utilized Python software for training, testing, and validation. We evaluated the performance of our proposed model using accuracy, precision, sensitivity, f1-score, and ROC-AUC score. The proposed models outperformed the conventional models and achieved 100% predictive accuracy. The proposed approach has the potential to be adapted as an intelligence-monitoring system for early medical diagnosis during pregnancy. The proposed techniques will help medical professionals make quick decisions accurately and enhance monitoring to improve the level of care offered to pregnant mothers and their unborn children.

A highly accurate and robust prediction framework for drilling rate of penetration based on machine learning ensemble algorithm

The rate of penetration (ROP) is a key indicator of drilling efficiency. Many researchers have explored the application of machine learning in ROP prediction. However, few studies have focused on the robustness of the constructed models, and developing a ROP prediction model that can achieve both high accuracy and strong robustness remains a challenge. This paper introduces a novel machine learning approach to constructing a ROP prediction model through ensemble learning algorithms. The model is based on field data from oilfields, incorporating 16 input parameters that influence ROP. First, the feasibility of the collected dataset is verified using correlation analysis. Then, ROP prediction models are developed based on various machine learning algorithms, including Decision Tree Regression (DTR), Random Forest (RF), eXtreme Gradient Boosting (XGB), Light Gradient Boosting Machine (LGBM), Support Vector Regression (SVR), and Backpropagation Neural Network (BPNN). By comparing the performance of these models under noise levels of 0%, 1.7%, and 5.1%, RF, LGBM, XGB, and SVR are selected as base learners. These base learners are then combined to construct multiple ensemble models, and the performance of the optimal ensemble model is evaluated under varying noise levels. The results show that the prediction error of the optimal model remains within 10%, and R2 is greater than 0.96. Finally, the Shapley Additive Explanations (SHAP) method is applied to perform interpretability analysis on the optimal ROP prediction model, examining the impact of different input factors on the model's predictive performance. Compared to single models and other ensemble models, the proposed ensemble model not only achieves higher accuracy but also demonstrates strong robustness and generalization capability.

Emotion recognition based on a limited number of multimodal physiological signals channels

Emotion constitutes a higher-order cognitive process, moreover, physiological signals can offer a more objective and realistically reflection of human emotional state. Electroencephalograph (EEG) signals, controlled by the central nervous system, are highly sensitive to emotional state fluctuations. Current research primarily focuses on recognizing emotions through feature extraction from multi-channel EEG signals. However, the acquisition of multi-channel EEG signals is hindered by difficulties acquiring from hair occlusion. The objective of this study is to construct a real-time physiological-emotion recognition model based on 2-channel EEG signals from the forehead without hair occlusion combined with 3-channel peripheral physiological (PPS) signals, employing signal preprocessing, feature extraction, feature normalization, and the ensemble learning algorithm Light Gradient Boosting Machine (LightGBM). In this paper, thirty participants sequentially viewed fourteen virtual reality (VR) scenes, rated for valence and arousal using the Self-Assessment Manikin (SAM) scale after each scene. The 2-channel EEG signals and 3-channel PPS signals were recorded synchronously from forehead by VR Head-Mounted Displays (HMD) Bio Pad developed in our research center. Our proposed model achieved median accuracy of 89.68% for valence, 90.11% for arousal in binary classification, and 84.93% for four classification of emotion recognition based on the collected samples. Furthermore, verification result of the proposed method on a public database (DEAP) achieved accuracy of 84.03%, 84.37%, and 72.23% on valence, arousal, and four classification, respectively. The proposed physiological-emotion recognition model, utilizing a limited number of multimodal physiological signals channels from wearable devices, demonstrates higher accuracy compared with the results in some literature, suggesting its potential for real-time implementation due to limited signal channels and low runtime.

Applicability of ensemble learning in total organic carbon and porosity evaluation of shales

Accurate evaluation of total organic carbon (TOC) content and porosity is of paramount significance for assessment and target interval selection for shale reservoirs. This study takes shales from the western Chongqing area as an exemplary case to delve into the applicability and reliability of ensemble learning in evaluating TOC content and porosity. The results indicate that although both Light Gradient Boosting Machine (LightGBM) and Random Forest (RF) algorithms are suitable for evaluating TOC content and porosity in shales, LightGBM algorithm is preferred due to its comprehensive advantages, including higher accuracy, stronger generalization capability, and faster operating speed. For TOC content evaluation, the four most important logging parameters identified by LightGBM and RF are consistent, but exhibit different orders: DEN (compensated density) > GR (gamma ray) > U (uranium) > CNL (compensated neutron) and DEN > U > GR > CNL, respectively. For porosity evaluation, LightGBM and RF identify the same type and order of the three most important logging parameters: AC (acoustic transit time) > DEN > U. This similarity may be attributed to the fact that both algorithms utilize Classification and Regression Tree (CART) as base learners. The dependence plots between SHAP (SHapley Additive exPlanations) values and logging parameters reveal that the role of each logging parameter in the evaluation model is segmented, rather than exhibiting a continuous linear contribution. In conclusion, given the exceptional performance of ensemble learning algorithms, they, especially LightGBM algorithm, are highly recommended for shale evaluation.

Recognizing Implicitly Toxic Content Based on Multiple Attention Mechanisms

ABSTRACTToxic content on social media has posed a significant threat to user experience and societal stability. In contrast to explicitly toxic content, implicitly toxic content, lacking overt toxic words, is challenging to identify through single‐text features. Therefore, this study proposes a multi‐feature fusion algorithm to recognize implicit toxic content. Firstly, we collect various features for each post, including text content, likes, comments, user information and other features. Subsequently, employing a multi‐head attention mechanism, we extract and fuse these features. Then, utilizing ensemble learning algorithms, we identify implicitly toxic content based on the fused features. Finally, we analyze the decision‐making process of the model using an interpretable algorithm and derive the most critical features and factors for identifying implicitly toxic content. The experimental results show that the proposed algorithm outperforms algorithms such as BERT, TextCNN, and XGBoost, demonstrating the advantage of multiple features over a single text feature in recognizing implicitly toxic content. In addition, this study provides insights into the decision‐making process of the model and provides more effective toxic content management strategies for social media platforms.

An Implementation of the Particle Flow Filter in an Atmospheric Model

Abstract The particle flow filter (PFF) shows promise for fully nonlinear data assimilation (DA) in high-dimensional systems. However, its application in atmospheric models has been relatively unexplored. In this study, we develop a new algorithm, PFF-DART, in order to conduct DA for high-dimensional atmospheric models. PFF-DART combines the PFF and the two-step ensemble filtering algorithm in the Data Assimilation Research Testbed (DART), exploiting the highly parallel structure of DART. To evaluate the performance of PFF-DART, we conduct an observing system simulation experiment (OSSE) in a simplified atmospheric general circulation model and compare the performance of PFF-DART with an existing linear and Gaussian DA method. Using the PFF-DART algorithm, we demonstrate, for the first time, the capability of the PFF to yield stable results in a yearlong cycling DA OSSE. Moreover, PFF-DART retains the important ability of the PFF to improve the assimilation of nonlinear and non-Gaussian observations. Finally, we emphasize that PFF-DART is a versatile algorithm that can be integrated with numerous other non-Gaussian DA techniques. This quality makes it a promising method for further investigation within a more sophisticated numerical weather prediction model in the future. Significance Statement Data assimilation is an important tool in weather forecasting. However, fundamental issues arising from linear and Gaussian approximations still exist, which can limit our utilization of observations. The particle flow filter is a promising method that avoids these approximations, but efficient algorithms for this method have yet to be developed. In this study, we develop an algorithm for the particle flow filter that can be implemented in high-dimensional geophysical models. We have demonstrated, for the first time, that the particle flow filter runs efficiently for a yearlong experiment in an atmospheric model. The new algorithm also improves the results over a commonly used data assimilation method. The results in this work demonstrate the potential usage of the particle flow filter in the weather forecasting and other high-dimensional forecasting problems.

A novel incremental ensemble learning for real-time explainable forecasting of electricity price

The development of a stable, safe, secure and sustainable energy future is a challenge for all countries these days. In terms of electricity price, its volatile nature makes its prediction a complex task. A precise real-time forecast of the electricity price can have significant consequences for the economy and risks faced. This work presents a new ensemble learning algorithm for making real-time predictions of electricity price in Spain. It combines long and short-term behavior patterns following an online incremental learning approach, keeping the model always up to date. The detection of novelties and unexpected behaviors in the time series streams allows the algorithm to provide more accurate predictions than the reference machine learning algorithms with which it is compared. In addition, the proposed algorithm predicts in real-time and the predictions obtained are interpretable, thus contributing to the Explainable Artificial Intelligence.

Enhanced drug classification using machine learning with multiplexed cardiac contractility assays

Cardiac screening of newly discovered drugs remains a longstanding challenge for the pharmaceutical industry. While therapeutic efficacy and cardiotoxicity are evaluated through preclinical biochemical and animal testing, 90 % of lead compounds fail to meet safety and efficacy benchmarks during human clinical trials. A preclinical model more representative of the human cardiac response is needed; heart tissue engineered from human pluripotent stem cell derived cardiomyocytes offers such a platform. In this study, three functionally distinct and independently validated engineered cardiac tissue assays are exposed to increasing concentrations of known compounds representing 5 classes of mechanistic action, creating a robust electrophysiology and contractility dataset. Combining results from six individual models, the resulting ensemble algorithm can classify the mechanistic action of unknown compounds with 86.2 % predictive accuracy. This outperforms single-assay models and offers a strategy to enhance future clinical trial success aligned with the recent FDA Modernization Act 2.0.

Toward data efficient anomaly detection in heterogeneous edge-cloud environments using clustered federated learning

Anomaly detection in edge-cloud scenarios stands as a critical means to ensure the security of network environment. Federated learning (FL)-based anomaly detection combines multiple data sources and ensures data privacy, making it a promising distributed detection method. However, FL-based anomaly detection system is usually affected by data heterogeneity and data bias, resulting in the inefficiency of data used for FL and the decline of detection performance. We propose an iterative federated clustering ensemble algorithm named IFCEA, in which we (1) establish a committee on the devices, and select the optimal participation for each device based on the evaluations of committee; (2) filter the clusters based on committee results, and exclude the biased clusters; (3) design an aggregation weight that reflects the degree of local distribution balance; (4) present a novel cluster initialization method, OneBiPartition, which adapts to the number of clusters and commences clustering federated task efficiently. IFCEA enhances the data quality used in FL-based anomaly detection system from two perspectives: device selection and participation weights, effectively addressing the issues of data heterogeneity and data bias faced during the FL training phase. Extensive experimental results on five network traffic datasets (the UNSW-NB15, CIC-IDS2017, CIC-IDS2018, CIC-DDoS2019 and BCCC-DDoS2024 datasets) demonstrate that our proposed framework outperforms in terms of detection metrics and convergence performance.

Ensemble and optimization algorithm in support vector machines for classification of wheat genotypes.

This study aimed to classifying wheat genotypes using support vector machines (SVMs) improved with ensemble algorithms and optimization techniques. Utilizing data from 302 wheat genotypes and 14 morphological attributes to evaluate six SVM kernels: linear, radial basis function (RBF), sigmoid, and polynomial degrees 1-3. Various optimization methods, including grid search, random search, genetic algorithms, differential evolution, and particle swarm optimization, were used. The radial basis function kernel achieves the highest accuracy at 93.2%, and the weighted accuracy ensemble further improves it to 94.9%. This study shows the effectiveness of these methods in agricultural research and crop improvement. Notably, optimization-based SVM classification, particularly with particle swarm optimization, saw a significant 1.7% accuracy gain in the test set, reaching 94.9% accuracy. These findings underscore the efficacy of RBF kernels and optimization techniques in improving wheat genotype classification accuracy and highlight the potential of SVMs in agricultural research and crop improvement endeavors.

Fake News Detection with Machine Learning Algorithms

Nowadays, with the advancement of technology, the use of news sources has also undergone a great evolution. News sources have constantly evolved from past to present, ranging from magazines to radios, from newspapers to televisions. The fact that it has become so easy to access news has caused society to pay more attention to fake news. Fake news has the ability to influence society through news sources such as social media, which can reach wider audiences with the development of technology. The difficulties of users in accessing accurate and reliable sources in this information flow that shapes their daily lives increases the potential for the spread of fake news, and it becomes increasingly difficult to distinguish between real and fake news. In this study, classification models for fake news detection were designed using machine learning algorithms. The dataset, which includes fake and real news examples, contains 42,000 examples. Each class, including fake and real samples, contains 22,000 sample data. In order to increase data quality, accuracy and usability, preprocessing methods were applied to the data set. The removal of numbers, stop words, and html tags was done in the pre-processing step to remove unnecessary information from the text. Models were created for fake news detection with singular and ensemble classification algorithms. Performance evaluation of the models was performed using 5-fold cross-validation. In the performance comparisons of the models, values such as accuracy, sensitivity, specificity, tp rate and fp rate were calculated. The highest performance results were observed in the random forest classification algorithm with an accuracy rate of 76%.

Diabetes prediction model based on GA-XGBoost and stacking ensemble algorithm.

Diabetes, as an incurable lifelong chronic disease, has profound and far-reaching effects on patients. Given this, early intervention is particularly crucial, as it can not only significantly improve the prognosis of patients but also provide valuable reference information for clinical treatment. This study selected the BRFSS (Behavioral Risk Factor Surveillance System) dataset, which is publicly available on the Kaggle platform, as the research object, aiming to provide a scientific basis for the early diagnosis and treatment of diabetes through advanced machine learning techniques. Firstly, the dataset was balanced using various sampling methods; secondly, a Stacking model based on GA-XGBoost (XGBoost model optimized by genetic algorithm) was constructed for the risk prediction of diabetes; finally, the interpretability of the model was deeply analyzed using Shapley values. The results show: (1) Random oversampling, ADASYN, SMOTE, and SMOTEENN were used for data balance processing, among which SMOTEENN showed better efficiency and effect in dealing with data imbalance. (2) The GA-XGBoost model optimized the hyperparameters of the XGBoost model through a genetic algorithm to improve the model's predictive accuracy. Combined with the better-performing LightGBM model and random forest model, a two-layer Stacking model was constructed. This model not only outperforms single machine learning models in predictive effect but also provides a new idea and method in the field of model integration. (3) Shapley value analysis identified features that have a significant impact on the prediction of diabetes, such as age and body mass index. This analysis not only enhances the transparency of the model but also provides more precise treatment decision support for doctors and patients. In summary, this study has not only improved the accuracy of predicting the risk of diabetes by adopting advanced machine learning techniques and model integration strategies but also provided a powerful tool for the early diagnosis and personalized treatment of diabetes.

Which classifiers are connected to others? An optimal connection framework for multi-layer ensemble systems

Ensemble learning is a powerful machine learning strategy that combines multiple models e.g. classifiers to improve predictions beyond what any single model can achieve. Until recently, traditional ensemble methods typically use only one layer of models which limits the exploration of different aspects in the classifiers’ predictions. On the other hand, the rise of deep learning has introduced multi-layer architectures that can learn complex functions by transforming data into multiple levels of representation. This characteristic of deep learning suggests that multi-layer ensembles may potentially provide better performance compared to single-layer ensembles. However, a problem which might arise is that in the subsequent layers, not all the inputs to a classifier are desirable, leading to lower performance. In this paper, we introduce a novel multi-layer ensemble of classifiers named COME in which each classifier at a specific layer is connected to multiple classifiers in the previous layer. These connections signify the use of the previous-layer-classifiers’ outputs as inputs for training the current layer's classifier. Each classifier can be connected to different classifiers in the previous layer, which allows inputs in each layer to be optimally selected. We propose a binary encoding scheme to encode the topology of the proposed multi-layer ensemble with defined connections between layers. Differential Evolution, a popular evolutionary computation method, is used as the optimisation algorithm to search for the optimal set of connections. Experimental results on 30 datasets from the UCI Machine Learning Repository and OpenML demonstrate that our proposed ensemble outperforms many state-of-the-art ensemble learning algorithms.

Prediction of Solvent Composition for Absorption-Based Acid Gas Removal Unit on Gas Sweetening Process.

The gas sweetening process is essential for removing harmful acid gases, such as hydrogen sulfide (H2S) and carbon dioxide (CO2), from natural gas before delivery to end-users. Consequently, chemical absorption-based acid gas removal units (AGRUs) are widely implemented due to their high efficiency and reliability. The most common solvent used in AGRU is monodiethanolamine (MDEA), often mixed with piperazine (PZ) as an additive to accelerate acid gas capture. The absorption performance, however, is significantly influenced by the solvent mixture composition. Despite this, solvent composition is often determined through trial and error in experiments or simulations, with limited studies focusing on predictive methods for optimizing solvent mixtures. Therefore, this paper aims to develop a predictive technique for determining optimal solvent compositions under varying sour gas conditions. An ensemble algorithm, Extreme Gradient Boosting (XGBoost), is selected to develop two predictive models. The first model predicts H2S and CO2 concentrations, while the second model predicts the MDEA and PZ compositions. The results demonstrate that XGBoost outperforms other algorithms in both models. It achieves R2 values above 0.99 in most scenarios, and the lowest RMSE and MAE values of less than 1, indicating robust and consistent predictions. The predicted acid gas concentrations and solvent compositions were further analyzed to study the effects of solvent composition on acid gas absorption across different scenarios. The proposed models offer valuable insights for optimizing solvent compositions to enhance AGRU performance in industrial applications.

Development and validation of a deep learning-based survival prediction model for pediatric glioma patients: A retrospective study using the SEER database and Chinese data

ObjectiveDevelop a time-dependent deep learning model to accurately predict the prognosis of pediatric glioma patients, which can assist clinicians in making precise treatment decisions and reducing patient risk. Study designThe study involved pediatric glioma patients from the Surveillance, Epidemiology, and End Results (SEER) Registry (2000–2018) and Tangdu Hospital in China (2010–2018) within specific time frames. For training, we selected two neural network-based algorithms (DeepSurv, neural multi-task logistic regression [N-MTLR]) and one ensemble learning-based algorithm (random survival forest [RSF]). Additionally, a multivariable Cox proportional hazard (CoxPH) model was developed for comparison purposes. The SEER dataset was randomly divided into 80 % for training and 20 % for testing, while the Tangdu Hospital dataset served as an external validation cohort. Super-parameters were fine-tuned through 1000 repeated random searches and 5-fold cross-validation on the training cohort. Model performance was assessed using the concordance index (C-index), Brier score, and Integrated Brier Score (IBS). Furthermore, the accuracy of predicting survival at 1, 3, and 5 years was evaluated using receiver operating characteristic (ROC) curves, calibration curves, and the area under the ROC curves (AUC). The generalization ability of the model was assessed using the C-index of the Tangdu Hospital data, ROC curves for 1, 3, and 5 years, and AUC values. Lastly, decision curve analysis (DCA) curves for 1, 3, and 5-year time frames are provided to assess the net benefits across different models. ResultsA total of 9532 patients with pediatric glioma were included in this study, comprising 9274 patients from the SEER database and 258 patients from Tangdu Hospital in China. The average age at diagnosis was 9.4 ± 6.2 years, and the average survival time was 96 ± 66 months. Through comprehensive performance comparison, the DeepSurv model demonstrated the highest effectiveness, with a C-index of 0.881 on the training cohort. Furthermore, it exhibited excellent accuracy in predicting the 1-year, 3-year, and 5-year survival rates (AUC: 0.903–0.939). Notably, the DeepSurv model also achieved remarkable performance and accuracy on the Chinese dataset (C-index: 0.782, AUC: 0.761–0.852). Comprehensive analysis of DeepSurv, N-MTLR, and RSF revealed that tumor stage, radiotherapy, histological type, tumor size, chemotherapy, age, and surgical method are all significant factors influencing the prognosis of pediatric glioma. Finally, an online version of the pediatric glioma survival predictor based on the DeepSurv model has been established and can be accessed through https://pediatricglioma-tangdu.streamlit.app. ConclusionsThe DeepSurv model exhibits exceptional efficacy in predicting the survival of pediatric glioma patients, demonstrating strong performance in discrimination, calibration, stability, and generalization. By utilizing the online version of the pediatric glioma survival predictor, which is based on the DeepSurv model, clinicians can accurately predict patient survival and offer personalized treatment options.

Classification of Quality Defects using Multivariate Control Chart with Ensemble Machine Learning Model

Multivariate control charts enable to monitor processes affected by more than one variable. But, when the process is out of control, it cannot detect which variable is causing it. It is an important requirement to know which variables in the process need corrective actions. In this study, a machine learning-based model is proposed to predict the variable/s that make the process out of control. For this purpose, ensemble algorithms, which are known to have higher prediction performance than single algorithms, were preferred. Because it is aimed to determine the variable(s) that cause the process to be out of control in the most accurate way. It is thought that a classification model in which ensemble algorithms are used together can increase the prediction accuracy. The model, which has not been encountered before in a quality control problem, was applied to a real problem and 98.06% classification accuracy was achieved. Another benefit is that it can predict the variable/variables that make the process uncontrolled without the need for multivariate control charts.

Spatial impulse response analysis and ensemble learning for efficient precision level sensing

In this paper, we propose an innovative method for determining the fill level of containers, such as trash cans, addressing a critical aspect of waste management. The method combines spatial impulse response analysis with machine learning (ML) techniques, offering a unique and effective approach for sound-based classification that can be extended to various domains beyond waste management. By employing a buzzer-generated sine sweep signal, we create a distinctive signature specific to the fill level of the waste container. This signature, once accurately decoded, is then interpreted by a specially developed ensemble learning algorithm. Our approach achieves a classification accuracy of over 90% when implemented locally on a development board, optimizing operational efficiencies and eliminating the need to delegate complex classification tasks to external entities. Using low-cost and energy-efficient hardware components, our method offers a cost-effective approach that contributes to sustainable and efficient waste management practices, providing a reliable and locally deployable solution.

Implementing Stacking with Cross - Validation for Heart Disease Prediction.

The objective of this work is to apply machine learning techniques for the prediction and early identification of cardiovascular disease, a major worldwide health problem. XGBoost, K-nearest neighbors (KNN), decision tree (DT), support vector machine (SVM), and StackingCVClassifier were among the ensemble algorithms used to anticipate cardiac disease using a dataset that came from the UCI ML database. To improve the quality of the data, an exploratory data analysis was performed on the dataset using methods including outlier identification and missing value imputation. Stacking CV Classifier attained the best accuracy rate of 92.20%, according to a comparative examination of pre-processing and post-processing findings. When compared to previous approaches, the suggested strategies performed better in terms of accuracy, recall, and f1-score. Moreover, the flexibility of the model indicates its possible application to other illnesses with comparable features.

Ensemble machine learning algorithm for cost-effective and timely detection of diabetes in Maiduguri, Borno State

Diabetes is a serious medical condition that severely hinders the body's ability to produce or properly regulate insulin, leading to detrimental carbohydrate metabolism and dangerously high blood sugar levels. This ultimately causes inadequate carbohydrate metabolism and heightened blood glucose levels. Alarmingly, from 2000 to 2019, diabetes-related mortality rates rose by 3%. In the year 2019 alone, diabetes was tragically responsible for nearly 2 million deaths. This groundbreaking research introduces the improved weighted average ensemble learning (WAEL) model as an innovative solution for detecting diabetes. The enhanced WAEL model effectively addresses the overfitting challenge by integrating multiple models that have gained unique insights from the data. The proposed WAEL model ingeniously combines five feature spaces through the grey wolf optimisation (GWO) algorithm to uncover the optimal weight combination. GWO plays a vital role in weight optimization, enabling the reduction of weights in models that are particularly sensitive to noise. The results demonstrated that the improved WAEL achieved an astounding level of accuracy, soaring to 98.90%. The LGBM algorithm followed closely, achieving an impressive accuracy of 85.00%. The RF method recorded an accuracy of 81.00%. When it comes to accurately identifying diabetes, the improved WAEL ensemble model significantly outperformed the other five individual models, as evidenced by metrics such as accuracy, precision, recall, and F1-score. Therefore, the proposed model stands as a compelling alternative tool for healthcare professionals in the early detection of diabetes.

Enhanced intrusion detection model based on principal component analysis and variable ensemble machine learning algorithm

The intrusion detection system (IDS) model, which can identify the presence of intruders in the network and take some predefined action for safe data transit across the network, is advantageous in achieving security in both simple and advanced network systems. Several IDS models have various security problems, such as low detection accuracy and high false alarms, which can be caused by the network traffic dataset's excessive dimensionality and class imbalance in the creation of IDS models. Principal Component Analysis (PCA) has proven to be a helpful feature selection technique for dimensionality reduction. As a result, because it is a linear transformation, it has challenges capturing non-linear relationships between feature properties in the network traffic datasets. This paper proposes a variable ensemble machine learning method to solve the problem and achieve a low variance model with high accuracy and low false alarm. First, PCA is combined with the AdaBoost ensemble machine learning algorithm, which acts as stagewise additive modelling to compensate for PCA's deficiency in feature selection in network traffic by minimizing the exponential loss function. Secondly, PCA is used for feature selection, and a LogitBoost classifier algorithm can be used for multiclass classification and acts as an additive tree regression to compensate for the PCA's weakness by minimizing the Logistic Loss to provide an optimal classifier output. Finally, the low variance ability of RandomForest, which employs the bagging approach, is applied to eliminate overfittings. The experiments of the IDS model developed from the proposed methods were evaluated on the WSN-DS, NSL-KDD, and UNSW-N15 datasets. The performance of the methods, PCA with AdaBoost, on the WSN-DS dataset has an accuracy score of 92.3 %, an 89.0 % accuracy score on the NSL-KDD dataset, and a 67.9 % accuracy score on UNSW-N15, which is the least accurate score. PCA and RandomForest surpassed them by scoring 100 % accuracy on all three datasets. PCA and Bagging have an accuracy score of 99.8 % on the WSN-DS dataset, 100 % on the NSL-KDD dataset, and 93.4 % on the UNSW-N15 dataset. In comparison, PCA and LogitBoost have an accuracy score of 98.9 % on the WSN-DS dataset, 100 % on the NSL-KDD dataset, and 88.7 % on the UNSW-N15 dataset.