Ensemble Techniques Research Articles

Infusing Weighted Average Ensemble Diversity for Advanced Breast Cancer Detection

ABSTRACTBreast cancer is a widespread health threat for women globally, often difficult to detect early due to its asymptomatic nature. As the disease advances, treatment becomes intricate and costly, ultimately resulting in elevated fatality rates. Currently, despite the widespread use of advanced machine learning (ML) and deep learning (DL) techniques, a comprehensive diagnosis of breast cancer remains elusive. Most of the existing methods primarily utilize either attention‐based deep models or models based on handcrafted features to capture and gather local details. However, both of these approaches lack the capability to offer essential local information for precise tumor detection. Additionally, the available breast cancer datasets suffer from class imbalance issue. Hence, this paper presents a novel weighted average ensemble network (WA‐ENet) designed for early‐stage breast cancer detection that leverages the ability of ensemble technique over single classifier‐based models for more robust and accurate prediction. The proposed model employs a weighted average‐based ensemble technique, combining predictions from three diverse classifiers. The optimal combination of weights is determined using the hill climbing (HC) algorithm. Moreover, the proposed model enhances overall system performance by integrating deep features and handcrafted features through the use of HOG, thereby providing precise local information. Additionally, the proposed work addresses class imbalance by incorporating borderline synthetic minority over‐sampling technique (BSMOTE). It achieves 99.65% accuracy on BUSI and 97.48% on UDIAT datasets.

An automated ensemble approach using Harris Hawk optimization for visually evoked EEG signal classification.

Steady-state visually evoked potential is one of the active explorations in the brain-computer interface research. Electroencephalogram based brain computer interface studies have been widely applied to perceive solutions for real-world problems in the healthcare domain. The classification of externally bestowed visual stimuli of different frequencies on a human was experimented to identify the need of paralytic people. Although many classifiers are at the fingertip of machine learning technology, recent research has proven that ensemble learning is more efficacious than individual classifiers. Despite its efficiency, ensemble learning technology exhibits certain drawbacks like taking more time on selecting the optimal classifier subset. This research article utilizes the Harris Hawk Optimization algorithm to select the best classifier subset from the given set of classifiers. The objective of the research is to develop an efficient multi-classifier model for electroencephalogram signal classification. The proposed model utilizes the Boruta Feature Selection algorithm to select the prominent features for classification. Thus selected prominent features are fed into the multi-classifier subset which has been generated by the Harris Hawk Optimization algorithm. The results of the multi-classifier ensemble model are aggregated using Stacking, Bagging, Boosting, and Voting. The proposed model is evaluated against the acquired dataset and produces a promising accuracy of 96.1%, 98.7%, 91.91%, and 99.01% with the ensemble techniques respectively. The proposed model is also validated with other performance metrics such as sensitivity, specificity, and F1-Score. The experimental results show that the proposed model proves its supremacy in segregating the multi-class classification problem with high accuracy.

Machine Learning in Cybersecurity: Advanced Detection and Classification Techniques for Network Traffic Environments

In the digital age, the integrity of business operations and the smoothness of their execution heavily depend on cybersecurity and network efficiency. The need for robust solutions to prevent cyber threats and enhance network functionality has never been more critical. This research aims to utilize machine learning (ML) techniques for the meticulous analysis of network traffic, with the dual goals of detecting anomalies and categorizing network activities to bolster security and performance. Employing a detailed methodology, this study begins with data preparation and progresses through to the deployment of advanced ML models, including logistic regression, decision trees, and ensemble learning techniques. This approach ensures the accuracy of the analysis and facilitates a nuanced understanding of network dynamics. Our findings indicate a notable enhancement in identifying network inefficiencies and in the more accurate classification of network traffic. The application of ML models significantly reduces network delays and bottlenecks by providing a strong defence strategy against cyber threats and network shortcomings, thereby improving user satisfaction, and boosting the organizational reputation as a secure and effective service layer. Conclusively, the research highlights the pivotal role of machine learning in network traffic analysis, offering innovative insights and fresh perspectives on anomaly detection and the identification of malicious activities. It lays a foundation for future explorations and acts as an evaluation benchmark in the fields of cybersecurity and network management.

Development of explainable recognition models for ADMFB flow regimes based on ensemble learning

The spatiotemporal characteristics of fluidization within an Air Dense Medium Fluidized Bed (ADMFB) are critical for its heat and mass transfer, as well as for mixing and separation processes. Yet, detailed analyses of the dynamic behavior of flow regimes during progressive transformations are scarce. This study captured synchronous pressure drop signals at multiple points along the sidewall of the bed using a non-intrusive measurement technique. Statistical analyses were utilized to extract the pressure drop's characteristic parameters and examine the statistical features' spatiotemporal fluctuations during fluidization state transitions. We proposed a method to quantitatively characterize the bed's real-time fluidization state through the combined use of multiple-point pressure characteristics. Following this approach, a multidimensional spatio-temporal matrix was constructed to depict the flow regime transitions quantitatively. Concurrently, ensemble learning techniques were applied to classify flow regimes based on various feature parameters and sample lengths. In the final phase, comprehensive interpretations and detailed visual analyses were deployed on the inputs to the optimized ensemble learning model, XGBoost, utilizing the Shapley Additive Explanations (SHAP) framework. This study provides additional insights and methodologies for managing the transition and intelligent control of flow regimes within an ADMFB.

Reduced-order modeling of unsteady fluid flow using neural network ensembles

The use of deep learning has become increasingly popular in reduced-order models (ROMs) to obtain low-dimensional representations of full-order models. Convolutional autoencoders (CAEs) are often used to this end as they are adept at handling data that are spatially distributed, including solutions to partial differential equations. When applied to unsteady physics problems, ROMs also require a model for time-series prediction of the low-dimensional latent variables. Long short-term memory (LSTM) networks, a type of recurrent neural network useful for modeling sequential data, are frequently employed in data-driven ROMs for autoregressive time-series prediction. When making predictions at unseen design points over long time horizons, error propagation is a frequently encountered issue, where errors made early on can compound over time and lead to large inaccuracies. In this work, we propose using bagging, a commonly used ensemble learning technique, to develop a fully data-driven ROM framework referred to as the CAE-eLSTM ROM that uses CAEs for spatial reconstruction of the full-order model and LSTM ensembles for time-series prediction. When applied to two unsteady fluid dynamics problems, our results show that the presented framework effectively reduces error propagation and leads to more accurate time-series prediction of latent variables at unseen points.

Breast cancer relapse disease prediction improvements with ensemble learning approaches

Diagnosis and prognosis are especially difficult areas of medical research related to cancer due to the high incidence of breast cancer, which has surpassed all other cancers in terms of female mortality. Another factor that has a substantial influence on the quality of life of cancer patients is the fear that they may experience a relapse of their disease. The objective of the study is to give medical practitioners a more effective strategy for using ensemble learning techniques to forecast when breast cancer may recur. This research aimed to investigate the usage of deep neural networks (DNNs) and artificial neural networks (ANNs) in addition to machine learning (ML) based approaches, including bagging, averaging, and voting, to enhance the efficacy of breast cancer relapse diagnosis on two breast cancer relapse datasets. Results from the empirical study demonstrate that the proposed ensemble learning-enabled approach improves accuracies by 96.31% and 95.81%, precisions by 96.70% and 96.15%, sensitivities by 98.88% and 98.68%, specificities by 84.62% in both, F1-scores by 97.78% and 97.40%, and area under the curve (AUCs) of 0.987 and 0.978, with University Medical Centre, Institute of Oncology (UMCIO) and Wisconsin prognostic breast cancer (WPBC) datasets respectively. Consequently, these improved disease outcomes may encourage physicians to use this model to make better treatment choices.

Quantitative prediction of soil chromium content using laboratory-based visible and near-infrared spectroscopy with different ensemble learning models

Soil heavy metal chromium pollution poses significant threats to human health and ecosystems, necessitating accurate quantitative prediction methods for effective monitoring and management. This study aims to develop robust predictive models for soil chromium content in farmland soils of Mojiang Hani Autonomous County, Pu’er City, Yunnan Province, China. These models utilize ensemble learning techniques based on visible and near-infrared spectroscopy. Operations before model building involved partitioning datasets with the Kennard-Stone algorithm to ensure representative training and testing sets. Visible and near-infrared spectroscopy-data preprocessing was performed using Savitzky-Golay smoothing and first-order derivative transformations to enhance signal quality. Bands selection was achieved through the Successive Projections Algorithm (SPA), effectively reducing data dimensionality and collinearity. Six ensemble learning models were constructed and assessed for their predictive performance: Bagging-DTR, Random Forest (RF), Adaboost-DTR, XGBoost-DTR, Stacking-1, and Stacking-2. These models utilized Decision Trees (DTR) and Linear Regression (LR) as base learners. Results demonstrated that ensemble models significantly outperformed individual base learners. Notably, the Stacking-2 model achieved the highest accuracy with an R^2 of 0.954, RMSE of 125.967 mg/kg, and RPD of 4.667. To validate the model’s practical applicability, spatial interpolation of soil Cr content was conducted using the Kriging method based on Stacking-2 model predictions. The spatial distribution maps of measured and predicted values exhibited high congruence, underscoring the model’s effectiveness in accurately mapping Cr distribution across the study area. This study underscores the efficacy of integrating ensemble learning with visible and near-infrared spectroscopy-data preprocessing and SPA for precise soil heavy metal prediction. The findings offer valuable insights and a scientific basis for enhanced soil quality monitoring, environmental risk assessment, and informed agricultural land management and pollution control.

The object detection model uses combined extraction with KNN and RF classification

Object detection plays an important role in various fields. Developing detection models for 2D objects that experience rotation and texture variations is a challenge. In this research, the initial stage of the proposed model integrates the gray-level co-occurrence matrix (GLCM) and local binary patterns (LBP) texture feature extraction to obtain feature vectors. The next stage is classifying features using k-nearest neighbors (KNN) and random forest (RF), as well as voting ensemble (VE). System testing used a dataset of 4,437 2D images, the results for KNN accuracy were 92.7% and F1-score 92.5%, while RF performance was lower. Although GLCM features improve performance on both algorithms, KNN is more consistent. The VE approach provides the best performance with an accuracy of 93.9% and an F1-score of 93.8%, this shows the effectiveness of the ensemble technique in increasing object detection accuracy. This study contributes to the field of object detection with a new approach combining GLCM and LBP as feature vectors as well as VE for classification.

Enhanced content-based fashion recommendation system through deep ensemble classifier with transfer learning

With the rise of online shopping due to the COVID-19 pandemic, Recommender Systems have become increasingly important in providing personalized product recommendations. Recommender Systems face the challenge of efficiently extracting relevant items from vast data. Numerous methods using deep learning approaches have been developed to classify fashion images. However, those models are based on a single model that may or may not be reliable. We proposed a deep ensemble classifier that takes the probabilities obtained from five pre-trained models such as MobileNet, DenseNet, Xception, and the two varieties of VGG. The probabilities obtained from the five pre-trained models are then passed as inputs to a deep ensemble classifier for the prediction of the given item. Several similarity measures have been studied in this work and the cosine similarity metric is used to recommend the products for a classified product given by a deep ensemble classifier. The proposed method is trained and validated using benchmark datasets such as Fashion product images dataset and Shoe dataset, demonstrating superior accuracy compared to existing models. The results highlight the potential of leveraging transfer learning and deep ensemble techniques to enhance fashion recommendation systems. The proposed model achieves 96% accuracy compared to the existing models.

Optimasi Algoritma Machine Learning Menggunakan Teknik Bagging Pada Klasifikasi Diagnosis Kanker Payudara

Classification algorithms have a very important role in Machine Learning, but not all algorithms have the same performance in every case. Algorithm performance can be affected by the type of data used, differences in problem characteristics, and the parameters used. Additionally, ensemble learning techniques such as Bagging can affect algorithm performance. Therefore, the problem arises of how to choose the most suitable algorithm for a particular classification task and how to optimize the performance of the algorithm. This research aims to carry out a comparative analysis and optimization of classification algorithms in Machine Learning. Classification algorithms that will be evaluated include Support Vector Machine (SVM), Neural Network, Logistic Regression, Decision Tree, and K-Nearest Neighbors (K-NN). Evaluation of the performance of these algorithms will be carried out using the confusion matrix, Receiver Operating Characteristic (ROC) Curve, and Area Under Curva (AUC). The result of this research is a comparative analysis of the optimization of classification algorithms using the bagging technique. After carrying out the evaluation process using the confusion matrix and ROC curve, it was found that the algorithm optimization using the bagging technique only had an effect on the Decision Tree (DT) and K-Nearest Neighbors (KNN) algorithms. . The accuracy of the DT algorithm increased by 0.6% while the accuracy of KNN increased by 1.3%. The AUC value for the DT algorithm increased by 1.4% and the KNN algorithm increased by 0.3%.

Predicting the Risk of Myocardial Infarction (MI) using Machine Learning (ML)

The purpose of this study was to use machine learning to forecast the likelihood of a myocardial infarction and identify when one would occur. The study made use of a preprocessed dataset regarding cardiac attacks from Kaggle. The purpose of this study was to use machine learning to forecast the likelihood of a heart attack and identify when one would occur. The research used a preprocessed dataset from Kaggle related to heart attacks. K-nearest neighbors (KNN), Support Vector Classifier (SVC), Logistic Regression, Decision Tree, Naive Bayes, XGBoost, Random Forest, and Gradient Boosting were the eight techniques used in the study. According to the findings, the models that predicted heart attacks in our dataset the best were Decision Tree and Gradient Boosting. These models showed excellent precision, recall, and F1-score balance, among other important criteria. They can effectively reduce overfitting and generalize well to new data thanks to their ability to handle complicated, non-linear interactions and their use of regularization and ensemble learning techniques. Decision Tree and Gradient Boosting are the most reliable options for this predictive task because of their all-encompassing strengths, even if models like XGBoost and Random Forest also performed well while Logistic Regression and SVC produced strong results.

Utilizing an Ensemble of Extra Tree Model for Classifying Mesothelioma Cancer

Objectives: Explore the potential of ensemble learning techniques like Bagging Tree, Random Forest, and Ensemble Extra Tree in transforming mesothelioma diagnosis.Overcome challenges associated with late-stage detection and limited treatment options using advanced machine learning algorithms.Enhance predictive power and feature extraction capabilities through the combination of diverse ensemble algorithms.

Long Short-Term Renewable Energy Sources Prediction for Grid-Management Systems Based on Stacking Ensemble Model

The transition towards sustainable energy systems necessitates effective management of renewable energy sources alongside conventional grid infrastructure. This paper presents a comprehensive approach to optimizing grid management by integrating Photovoltaic (PV), wind, and grid energies to minimize costs and enhance sustainability. A key focus lies in developing an accurate scheduling algorithm utilizing Mixed Integer Programming (MIP), enabling dynamic allocation of energy resources to meet demand while minimizing reliance on cost-intensive grid energy. An ensemble learning technique, specifically a stacking algorithm, is employed to construct a robust forecasting pipeline for PV and wind energy generation. The forecasting model achieves remarkable accuracy with a Root Mean Squared Error (RMSE) of less than 0.1 for short-term (15 min and one day ahead) and long-term (one week and one month ahead) predictions. By combining optimization and forecasting methodologies, this research contributes to advancing grid management systems capable of harnessing renewable energy sources efficiently, thus facilitating cost savings and fostering sustainability in the energy sector.

Corrigendum: A data-driven ensemble technique for the detection of false data injection attacks in the smart grid framework

Corrigendum: A data-driven ensemble technique for the detection of false data injection attacks in the smart grid framework

Evaluating and Comparing AI Models for Hourly Energy Demand Prediction

This study investigates the application of various AI models to predict energy demand, comparing the performance of four specific models: Decision Tree, Random Forest, Gradient Boosting, and Linear Regression. The evaluation of these models' prediction performance reveals that ensemble methods like Random Forest and Gradient Boosting exhibit promising generalization capabilities, while the Decision Tree model shows high training accuracy but suffers from overfitting. The discussion underscores the importance of ensemble techniques and feature engineering optimization in mitigating overfitting and enhancing forecast accuracy. Furthermore, the study highlights the potential of AI-driven approaches to promote sustainability and resilience in energy systems, emphasizing the need for further optimization and collaboration among stakeholders to achieve a cleaner, more sustainable energy future.

Ensemble learning approach to enhancing binary classification in Intrusion Detection System for Internet of Things

The Internet of Things (IoT) has experienced significant growth and plays a crucial role in daily activities. However, along with its development, IoT is very vulnerable to attacks and raises concerns for users. The Intrusion Detection System (IDS) operates efficiently to detect and identify suspicious activities within the network. The primary source of attacks originates from external sources, specifi-cally from the internet attempting to transmit data to the host network. IDS can identify unknown attacks from network traffic and has become one of the most effective network security. Classification is used to distinguish between normal class and attacks in binary classification problem. As a result, there is a rise in the false positive rates and a decrease in the detection accuracy during the model's training. Based on the test results using the ensemble technique with the ensemble learning XGBoost and LightGBM algorithm, it can be concluded that both binary classification problems can be solved. The results using these ensemble learning algorithms on the ToN IoT Dataset, where binary classification has been performed by combining multiple devices into one, have demonstrated improved accuracy. Moreover, this ensemble approach ensures a more even distribution of accuracy across each device, surpassing the findings of previous research.

Modeling the strength parameters of agro waste-derived geopolymer concrete using advanced machine intelligence techniques

Abstract The mechanical strength of geopolymer concrete incorporating corncob ash and slag (SCA-GPC) was estimated by means of three distinct AI methods: a support vector machine (SVM), two ensemble methods called bagging regressor (BR), and random forest regressor (RFR). The developed models were validated using statistical tests, absolute error assessment, and the coefficient of determination (R 2). The importance of various modeling factors was determined by means of interaction diagrams. When estimating the flexural strength and compressive strength of SCA-GPC, R 2 values of over 0.85 were measured between the actual and predicted findings using both individual and ensemble AI models. Statistical testing and k-fold analysis for error evaluation revealed that the RFR model outperformed the SVM and BR models in terms of accuracy. As demonstrated by the interaction graphs, the mechanical characteristics of SCA-GPC were found to be extremely responsive to the mix proportions of ground granulated blast furnace slag, fine aggregate, and corncob ash. This was the case for all three components. This study demonstrated that highly precise estimations of mechanical properties for SCA-GPC can be made using ensemble AI techniques. Improvements in geopolymer concrete performance can be achieved by the implementation of such practices.

A Time-Domain Doppler Estimation and Waveform Recovery Approach with Iterative and Ensemble Techniques for Bi-Phase Code in Radar Systems

This paper presents a novel, cost-effective technique for estimating the Doppler effect in the time domain using a single pulse and subsequently leveraging the precise Doppler value to recover the radar waveform. The proposed system offers several key advantages over existing techniques, including the ability to calculate the target speed without any frequency ambiguity and the ability to detect a wide range of target speeds. These two features are not available in any existing techniques, including the conventional moving target detection (MTD) processor. To ensure improved accuracy and robust estimation, the system employs ensemble and iterative techniques by recursively and efficiently reducing the Doppler residues from the signal. Furthermore, the proposed system demonstrates effective signal recovery of a well-known bi-phase code shape at low signal-to-noise ratios in just a few iterations. The performance evaluation of the new algorithm demonstrates its practicability and its superiority over traditional radar systems. Implementation on software-defined radio (SDR) reveals that the proposed system excels in Doppler estimation and signal recovery at low SNRs, demonstrating promising results.

Advanced Deep Learning Methodologies in the Diagnosis of Parkinson's Disease: A Comprehensive Review

This study of the literature explores the field of sophisticated deep techniques for diagnosing Parkinson's illness and severity evaluation. The research looks into the use of machine learning algorithms as potential markers of Parkinson's illness in manual illustrations and speech impairments. These algorithms include XGBoost, Neural Networks with Recurrence, and ensemble models. Numerous feature extraction techniques, model comparison assessments, and preprocessing methodologies are investigated in an effort to improve precision and efficacy of Parkinson's illness diagnosis. In order to improve performance and interpretability of the prototype, the importance of feature engineering, ensemble learning techniques, and interpretability techniques like LIME and SHAP values is emphasized. The study also highlights how crucial it is to integrate several models for real-time monitoring systems to assist in the prompt identification and ongoing tracking of Parkinson's illness. Future possibilities for examine include adding other symptoms including handwriting distortion and olfactory sound loss, investigating new algorithms, and using deep learning approaches to improve system performance. Considerable progress in Parkinson's disease diagnosis and therapy can be made by filling up these research gaps and utilizing deep learning. By addressing these problems via additional study and technical developments, PD can be detected and managed earlier and more effectively, which will ultimately improve patient quality of life. Keywords: Keywords: interpretability, early diagnosis, severity assessment, ensemble learning, XGBoost, deep learning, recurrent neural networks,, machine learning, Parkinson's illness, and feature engineering.

Enhancing trustworthiness among iot network nodes with ensemble deep learning-based cyber attack detection

A lot of machine learning methods and expert systems are used in network intrusion detection automation. When different industrial control systems merge with the Internet of Things (IoT) environment, they become vulnerable to cyber-attacks in critical infrastructure situations requiring communication technologies. Conventional machine learning techniques used for network anomaly detection are ineffective due to the substantial amount of network traffic within important Cyber-Physical Systems (CPSs). In this manuscript, Cyberattack Identification Through Ensemble Deep Learning in an IoT environment is proposed. Initially, the input network traffic data are taken from the IoT-23 dataset. Then the network traffic data are preprocessed using Z-Score normalization to reduce any irrelevant or erroneous data from the input dataset. Then the relevant features are selected using the Gorilla Troops Optimization (GTO) algorithm. Afterwards, the selected features are fed into the ensemble classification model based on Random Space (RS), Random Tree (RT), Extreme Gradient Boosting (XGBoost), and Graph Convolutional Neural Network (GCNN). Among the several ensembling techniques, GCNN can achieve better performance. Python is used to accomplish the suggested technique. The performance of the proposed GCNN method provides 12.09%, 4.34%, and 3.21% higher accuracy than the other models like RS, RT and XGBoost respectively.