Ensemble Classifier Research Articles

A new chiller fault diagnosis method under the imbalanced data environment via combining an improved generative adversarial network with an enhanced deep extreme learning machine

The existing chiller fault diagnosis approaches often ignore the problem of data imbalance of chiller, which leads to low accuracy in diagnosing minority class fault samples. To conquer this issue, this paper proposes an improved generative adversarial network (IGAN) with an enhanced deep extreme learning machine (EDELM) method. Firstly, to better learn the latent structure of chiller fault data, the multi-head attention (MHA) mechanism is integrated into the traditional generative adversarial network (GAN) method to generate new samples that are more in line with the distribution of minority class fault samples for the purpose of obtaining a rebalanced dataset. Secondly, to fully handle the nonlinear features hidden in the massive chiller data, the deep extreme learning machine (DELM) basic classifier is trained on the rebalanced dataset. To enhance more attention to the misclassified samples, the adaptive boosting (AdaBoost) ensemble strategy is employed to train multiple DELM basic classifiers by updating the sample weights following the classification results through the iterative rounds. The voting weight of the current DELM basic classifier is given according to its fault diagnosis accuracy. Finally, multiple DELM basic classifiers are ensembled according to their voting weights to obtain the final ensemble classifier. The pattern of the snapshot sample is determined through the weighted voting strategy. Detailed experimental results based on the research project 1043 (RP-1043) conducted by the American society of heating, refrigeration, and air conditioning engineers (ASHRAE) confirm the effectiveness of the proposed IGAN-EDELM approach under imbalanced data environments.

An intelligent solvent selection approach in carbon capturing process: A comparative study of machine learning multi-class classification models

Carbon capture is crucial for mitigating climate change and achieving global emissions reduction targets. Among various technologies, absorption-based methods using aqueous solvents are among the most common and efficient, offering high capture rates and retrofit capabilities for existing industrial facilities. However, selecting the optimal solvent is challenging due to its direct impact on the performance, efficiency, and cost-effectiveness of the carbon capture process. This research introduces a novel multi-class classification methodology for solvent selection in carbon capture. Leveraging a dataset of 656 data points, including features such as temperature, pressure, molar ratio, and CO2 solubility, the study employs nine machine learning algorithms—traditional models (Naïve Bayes, k-Nearest Neighbors, Decision Tree) and ensemble methods (Gradient Boosted Tree, Random Forest, Bagging, AdaBoost, Stacking, Voting)—to develop robust classification models. The results highlight the superior performance of the Stacking ensemble classifier, achieving 99.24 % accuracy, 0.76 % error rate, 99.04 % weighted mean recall, 98.08 % weighted mean precision, a 98.56 % F1 score, and a 0.9911 Cohen's Kappa coefficient on the test set. The effectiveness of ensemble techniques over traditional models underscores their ability to capture the complex relationships inherent in solvent selection. This approach streamlines the decision-making process for designers and engineers, enabling rapid identification of the most suitable solvent class based on operational conditions. Integrating this methodology into chemical engineering software could offer practical benefits, facilitating real-time solvent selection and optimization in industrial carbon capture, ultimately contributing to more sustainable and efficient processes.

Automatic Classification of Anomalous ECG Heartbeats from Samples Acquired by Compressed Sensing.

In this paper, a method for the classification of anomalous heartbeats from compressed ECG signals is proposed. The method operating on signals acquired by compressed sensing is based on a feature extraction stage consisting of the evaluation of the Discrete Cosine Transform (DCT) coefficients of the compressed signal and a classification stage performed by means of a set of k-nearest neighbor ensemble classifiers. The method was preliminarily tested on five classes of anomalous heartbeats, and it achieved a classification accuracy of 99.40%.

Predictive modelling of metabolic syndrome in Ghanaian diabetic patients: an ensemble machine learning approach

Abstract Objectives The burgeoning prevalence of cardiometabolic disorders, including type 2 diabetes mellitus (T2DM) and metabolic syndrome (MetS) within Africa is concerning. Machine learning (ML) techniques offer a unique opportunity to leverage data-driven insights and construct predictive models for MetS risk, thereby enhancing the implementation of personalised prevention strategies. In this work, we employed ML techniques to develop predictive models for pre-MetS and MetS among diabetic patients. Methods This multi-centre cross-sectional study comprised of 919 T2DM patients. Age, gender, novel anthropometric indices along with biochemical measures were analysed using BORUTA feature selection and an ensemble majority voting classification model, which included logistic regression, k-nearest neighbour, Gaussian Naive Bayes, Gradient boosting classification, and support vector machine. Results Distinct metabolic profiles and phenotype clusters were associated with MetS progression. The BORUTA algorithm identified 10 and 16 significant features for pre-MetS and MetS prediction, respectively. For pre-MetS, the top-ranked features were lipid accumulation product (LAP), triglyceride-glucose index adjusted for waist-to-height ratio (TyG-WHtR), coronary risk (CR), visceral adiposity index (VAI) and abdominal volume index (AVI). For MetS prediction, the most influential features were VAI, LAP, waist triglyceride index (WTI), Very low-density cholesterol (VLDLC) and TyG-WHtR. Majority voting ensemble classifier demonstrated superior performance in predicting pre-MetS (AUC = 0.79) and MetS (AUC = 0.87). Conclusion Identifying these risk factors reveals the complex interplay between visceral adiposity and metabolic dysregulation in African populations, enabling early detection and treatment. Ethical integration of ML algorithms in clinical decision-making can streamline identification of high-risk individuals, optimize resource allocation, and enable precise, tailored interventions.

A combined AraBERT and Voting Ensemble classifier model for Arabic sentiment analysis

For sentiment analysis of short texts (e.g. movie reviews, tweets, etc.), one approach is to build machine learning models that can determine their tones (positive, negative, neutral). However, these natural language processing (NLP) studies are missing when there is a lack of high-quality and large-scale training data for specific languages such as Arabic. In this paper, we present three machine learning models designed to classify sentiment Arabic tweets developed for a Kaggle competition. We present a Voting Ensemble classifier taking advantage of both character-level and word-level features. We also propose an AraBERT (Arabic Bidirectional Encoder Representations from Transformers) model with preprocessing using Farasa Segmenter. Finally, we combine these first two approaches as a third approach (Voting Ensemble classifier using AraBERT embeddings). Performance measures of results show improvement over previous efforts for all models. The third model exhibits strong performance with a 73.98% F-score score. The work presented here could be useful for future studies and for new Arabic sentiment analysis online services or competitions.

DDSBC: A Stacking Ensemble Classifier-Based Approach for Breast Cancer Drug-Pair Cell Synergy Prediction.

Breast cancer (BC) ranks as a leading cause of mortality among women worldwide, with incidence rates continuing to rise. The quest for effective treatments has led to the adoption of drug combination therapy, aiming to enhance drug efficacy. However, identifying synergistic drug combinations remains a daunting challenge due to the myriad of potential drug pairs. Current research leverages machine learning (ML) and deep learning (DL) models for drug-pair synergy prediction and classification. Nevertheless, these models often underperform on specific cancer types, including BC, as they are trained on data spanning various cancers without any specialization. Here, we introduce a stacking ensemble classifier, the drug-drug synergy for breast cancer (DDSBC), tailored explicitly for BC drug-pair cell synergy classification. Unlike existing models that generalize across cancer types, DDSBC is exclusively developed for BC, offering a more focused approach. Our comparative analysis against classical ML methods as well as DL models developed for drug synergy prediction highlights DDSBC's superior performance across test and independent datasets on BC data. Despite certain metrics where other methods narrowly surpass DDSBC by 1-2%, DDSBC consistently emerges as the top-ranked model, showcasing significant differences in scoring metrics and robust performance in ablation studies. DDSBC's performance and practicality position it as a preferred choice or an adjunctive validation tool for identifying synergistic or antagonistic drug pairs in BC, providing valuable insights for treatment strategies.

Machine learning based classifiers for dynamic and transient disturbance classification in smart microgrid system

The smart microgrid system should have the ability to rapidly detect and classify every type of disturbance that happens in the network to operate the protection scheme and maintain the power quality. Both dynamic and transient types of disturbances are considered in this study, and the classification of each type of disturbance has been done using different machine learning-based techniques. A novel ensemble classifier has been proposed after comparing the performance of the decision tree (DT), k nearest neighbor (KNN), support vector machine (SVM), and ensemble classifier. The performances of all the classifiers have been evaluated in terms of accuracy, sensitivity, confusion matrices, and receiver operating characteristics (ROC). A very satisfactory result has been obtained in the ensemble tree classification technique with 99.3% accuracy in 2.1 s of training time. Therefore, it can be suggested to identify and classify every transient as well as steady-state events of the microgrid network.

StackDPPred: Multiclass prediction of defensin peptides using stacked ensemble learning with optimized features

Host defense or antimicrobial peptides (AMPs) are promising candidates for protecting host against microbial pathogens for example bacteria, virus, fungi, yeast. Defensins are the type of AMPs that act as potential therapeutic drug agent and perform vital role in various biological process. Conventional Experiments to identify defensin peptides (DPs) are time consuming and expensive. Thus, the shortcomings of wet lab experiments are leveraged by computational methods to accurately predict the functional types of DPs. In this paper, we aim to propose a novel multi-class ensemble-based prediction model called StackDPPred for identifying the properties of DPs. The peptide sequences are encoded using split amino acid composition (SAAC), segmented position specific scoring matrix (SegPSSM), histogram of oriented gradients-based PSSM (HOGPSSM) and feature extraction based graphical and statistical (FEGS) descriptors. Next, principal component analysis (PCA) is used to select the best subset of attributes. After that, the optimized features are fed into single machine learning and stacking-based ensemble classifiers. Furthermore, the ablation study demonstrates the robustness and efficacy of the stacking approach using reduced features for predicting DPs and their families. The proposed StackDPPred method improves the overall accuracy by 13.41% and 7.62% compared to existing DPs predictors iDPF-PseRAAC and iDEF-PseRAAC, respectively on validation test. Additionally, we applied the local interpretable model-agnostic explanations (LIME) algorithm to understand the contribution of selected features to the overall prediction. We believe, StackDPPred could serve as a valuable tool accelerating the screening of large-scale DPs and peptide-based drug discovery process.

Improving Cleaning of Solar Systems through Machine Learning Algorithms

The study focuses on the importance of maintaining photovoltaic (PV) systems for optimal performance in sustainable energy generation. It highlights the impact of dust accumulation on reducing system efficiency and proposes a method to predict system performance, aiding in scheduling cleaning activities effectively. Two prediction models are developed: one using time-series prediction techniques (LSTM, ARIMA, SARIMAX) to forecast Performance Ratio (PR), and another employing ensemble voting classifiers (RF, Log, GBM) to predict the need for cleaning. The SARIMAX model performs best, achieving high accuracy in PR prediction (R2 = 92.12%), while the classification model accurately predicts cleaning needs (91%). The research provides valuable insights for improving maintenance strategies and enhancing the efficiency and sustainability of PV systems.

Predicting the Performance of Ensemble Classification Using Conditional Joint Probability

In many machine learning applications, there are many scenarios when performance is not satisfactory by single classifiers. In this case, an ensemble classification is constructed using several weak base learners to achieve satisfactory performance. Unluckily, the construction of the ensemble classification is empirical, i.e., to try an ensemble classification and if performance is not satisfactory then discard it. In this paper, a challenging analytical problem of the estimation of ensemble classification using the prediction performance of the base learners is considered. The proposed formulation is aimed at estimating the performance of ensemble classification without physically developing it, and it is derived from the perspective of probability theory by manipulating the decision probabilities of the base learners. For this purpose, the output of a base learner (which is either true positive, true negative, false positive, or false negative) is considered as a random variable. Then, the effects of logical disjunction-based and majority voting-based decision combination strategies are analyzed from the perspective of conditional joint probability. To evaluate the forecasted performance of ensemble classifier by the proposed methodology, publicly available standard datasets have been employed. The results show the effectiveness of the derived formulations to estimate the performance of ensemble classification. In addition to this, the theoretical and experimental results show that the logical disjunction-based decision outperforms majority voting in imbalanced datasets and cost-sensitive scenarios.

Schizophrenia diagnosis based on diverse epoch size resting-state EEG using machine learning.

Schizophrenia is a severe mental disorder that impairs a person's mental, social, and emotional faculties gradually. Detection in the early stages with an accurate diagnosis is crucial to remedying the patients. This study proposed a new method to classify schizophrenia disease in the rest state based on neurologic signals achieved from the brain by electroencephalography (EEG). The datasets used consisted of 28 subjects, 14 for each group, which are schizophrenia and healthy control. The data was collected from the scalps with 19 EEG channels using a 250 Hz frequency. Due to the brain signal variation, we have decomposed the EEG signals into five sub-bands using a band-pass filter, ensuring the best signal clarity and eliminating artifacts. This work was performed with several scenarios: First, traditional techniques were applied. Secondly, augmented data (additive white Gaussian noise and stretched signals) were utilized. Additionally, we assessed Minimum Redundancy Maximum Relevance (MRMR) as the features reduction method. All these data scenarios are applied with three different window sizes (epochs): 1, 2, and 5 s, utilizing six algorithms to extract features: Fast Fourier Transform (FFT), Approximate Entropy (ApEn), Log Energy entropy (LogEn), Shannon Entropy (ShnEn), and kurtosis. The L2-normalization method was applied to the derived features, positively affecting the results. In terms of classification, we applied four algorithms: K-nearest neighbor (KNN), support vector machine (SVM), quadratic discriminant analysis (QDA), and ensemble classifier (EC). From all the scenarios, our evaluation showed that SVM had remarkable results in all evaluation metrics with LogEn features utilizing a 1-s window size, impacting the diagnosis of Schizophrenia disease. This indicates that an accurate diagnosis of schizophrenia can be achieved through the right features and classification model selection. Finally, we contrasted our results to recently published works using the same and a different dataset, where our method showed a notable improvement.

HarmonyNet: Navigating hate speech detection

In the digital era, social media platforms have become central to communication across various domains. However, the vast spread of unregulated content often leads to the prevalence of hate speech and toxicity. Existing methods to detect this toxicity struggle with context sensitivity, accommodating diverse dialects, and adapting to varied communication styles. To tackle these challenges, we introduce an ensemble classifier that leverages the strengths of language models and traditional deep neural network architectures for more effective hate speech detection on social media. Our evaluations show that this hybrid approach outperforms individual models and exhibits robustness against adversarial attacks. Future efforts will aim to enhance the model’s architecture to further boost its efficiency and extend its capability to recognize hate speech across an even wider range of languages and dialects.

Modeling of the effect of transportation system accessibility on residential real estate prices: The case of Washington metropolitan area, USA

Deep learning accurate predictions of house prices are essential for prospective homeowners, investors, appraisers, and insurers. However, some studies lack accuracy as they overlook critical factors like accessibility and economic attributes that influence house prices. This paper aims to predict house prices by considering structural, locational, accessibility, and economic attributes, while also exploring the effect of accessibility on housing prices. The dataset contains 2,019,663 real estate transaction records from 1975 to 2018 in the Washington metropolitan area, obtained from the Zillow website. In this study, the accessibility index is calculated using Distance, Cumulative Opportunities, and Gravity measures, with the gravity measure surpassing others due to its consideration of both land use and transportation aspects. Economic attributes are then utilized to predict the average monthly house price using deep learning algorithms such as LSTM, GRU, and Simple RNN, with the Simple RNN demonstrating superior performance. Following the amalgamation of structural and locational attributes with the accessibility index and average house prices, various machine learning algorithms—including Linear Regression, Lasso, Ridge, Random Forest, GBM, LightGBM, XGBoost, Decision Tree, AdaBoost, Artificial Neural Network, and Stacked Generalization—are employed for prediction. Subsequent evaluation reveals that Stacked Generalization (ANN + LightGBM) provides the best performance, with an R-squared value of 0.96 and RMSE of $23,290. Moreover, this paper identifies accessibility index thresholds (80,003 for large buildings and 160,103 for small buildings) and demonstrates that a higher accessibility index leads to lower housing prices, attributed to noise pollution, decreased privacy, and increased supply responses.

Improved Methodology for Breast Cancer Prediction through Integration of Hard Voting Ensemble Classifier on WDBC Data Set

Improved Methodology for Breast Cancer Prediction through Integration of Hard Voting Ensemble Classifier on WDBC Data Set

Predicting non-chemotherapy drug-induced agranulocytosis toxicity through ensemble machine learning approaches.

Agranulocytosis, induced by non-chemotherapy drugs, is a serious medical condition that presents a formidable challenge in predictive toxicology due to its idiosyncratic nature and complex mechanisms. In this study, we assembled a dataset of 759 compounds and applied a rigorous feature selection process prior to employing ensemble machine learning classifiers to forecast non-chemotherapy drug-induced agranulocytosis (NCDIA) toxicity. The balanced bagging classifier combined with a gradient boosting decision tree (BBC + GBDT), utilizing the combined descriptor set of DS and RDKit comprising 237 features, emerged as the top-performing model, with an external validation AUC of 0.9164, ACC of 83.55%, and MCC of 0.6095. The model's predictive reliability was further substantiated by an applicability domain analysis. Feature importance, assessed through permutation importance within the BBC + GBDT model, highlighted key molecular properties that significantly influence NCDIA toxicity. Additionally, 16 structural alerts identified by SARpy software further revealed potential molecular signatures associated with toxicity, enriching our understanding of the underlying mechanisms. We also applied the constructed models to assess the NCDIA toxicity of novel drugs approved by FDA. This study advances predictive toxicology by providing a framework to assess and mitigate agranulocytosis risks, ensuring the safety of pharmaceutical development and facilitating post-market surveillance of new drugs.

Application of Deep Learning in the Classification of EEG Signals of Parkinson's Disease Patients

This article summarizes research on classifying electroencephalogram (EEG) signals of Parkinson's disease (PD) patients using deep learning technologies. Parkinson's disease is a common neurodegenerative disorder that severely affects quality of life. EEG signals provide valuable information about brain activity. Therefore, analyzing and classifying the EEG signals of PD patients can aid in early diagnosis and treatment. The paper outlines the assessment metrics and cross-validation methods in the diagnosis of Parkinson's disease, highlighting the effectiveness of deep learning in diagnosis. The key technologies reviewed in the article include Wavelet Packet Transform (WPT) and Deep Residual Neural Networks (DRSN), with the WPT-DRSN method achieving up to 99.92% prediction accuracy in the binary classification task of PD patients. Furthermore, this method has also been applied to more complex classweweification tasks, such as categorizing PD patients, REM sleep behavior disorder patients, and healthy controls, maintaining high accuracy. When analyzing Parkinson's disease EEG with the CRNN model, the accuracy is high, but the sample size is small and the model interpretability is limited, necessitating further validation and improvement. For EEG-based PD diagnosis and classification methods, high accuracy was achieved through ICA preprocessing, CSP feature extraction, and ensemble learning classifiers. Future research should consider the integration of multimodal data fusion, deep learning, and NLP technologies, as well as the development of personalized models to enhance the accuracy of PD diagnosis.

Fault classification scheme for TCSC compensated transmission line using MODWPT-BTEC

This article proposes a fault classification algorithm for a Thyristor Controlled Series Compensated (TCSC) long transmission line. Such type of line imposes issues related to reactance modulation, voltage inversion, current inversion and TCSC control action. This scheme utilizes ½ cycle post-fault current signal, which is then processed by maximum overlap discrete wavelet packet transform (MODWPT), and energy feature is calculated. The proposed scheme has been validated on a modified Western System Coordinating Council (WSCC) 9 bus system with TCSC. Also, the scheme is certified for wind farm integration in grid connected mode. The bagged tree ensemble classifier outperformed various stable and ensemble combinations, including the decision tree, linear discriminative analysis, support vector machine, k-nearest neighbor, boosted/trees ensemble, and subspace/discriminant ensemble. The line with TCSC compensation is modeled using PSCAD/EMTDC software. The proposed scheme accurately classifies faults in all tested scenarios with 100% accuracy. The outcomes of the suggested algorithm along with a comparative study confirm the accuracy and fast protection measures for TCSC-compensated line.

An optimized approach for detection and mitigation of DDoS attack cloud using an ensembled deep learning approach

An optimized approach for detection and mitigation of DDoS attack cloud using an ensembled deep learning approach

Highly accurate brain tumor detection with high sensitivity using transform-based functions and machine learning algorithms.

Brain tumor is an extremely dangerous disease with a very high mortality rate worldwide. Detecting brain tumors accurately is crucial due to the varying appearance of tumor cells and the dimensional irregularities in their growth. This poses a significant challenge for detection algorithms. Currently, there are numerous algorithms utilized for this purpose, ranging from transform-based methods to those rooted in machine learning techniques. These algorithms aim to enhance the accuracy of detection despite the complexities involved in identifying brain tumor cells. The major limitation of these algorithms is the mapping of extracted features of a brain tumor in the classification algorithms. To employ a combination of transform methods to extract texture feature from brain tumor images. This paper employs a combination of transform methods based on sub band decomposition for texture feature extraction from MRI scans, hybrid feature optimization methods using firefly and glow-worm algorithms for selection of feature, employment of MKSVM algorithm and stacking ensemble classifier for classification and application of the feature of fusion of different feature extraction methods. The algorithm under consideration has been put into practice using MATLAB, utilizing datasets from BRATS (Brain Tumor Segmentation) for the years 2013, 2015, and 2018. These datasets serve as the foundation for testing and validating the algorithm's performance across different time periods, providing a comprehensive assessment of its effectiveness in detecting brain tumors. The proposed algorithm achieves maximum detection accuracy, detection sensitivity and specificity up to 98%, 99% and 99.5% respectively. The experimental outcomes showcase the efficiency of the algorithm in detection of brain tumor. The proposed work mainly contributes in brain tumor detection in the following aspects: a) use of combination of transform methods for texture feature extraction from MRI scans b) hybrid feature selection methods using firefly and glow-worm optimization algorithms for selection of feature c) employment of MKSVM algorithm and stacking ensemble classifier for classification and application of the feature of fusion of different feature extraction methods.

General process detection through side channel characterization

ABSTRACT In this study, we demonstrate the feasibility of using physical sensors in System-on-a-Chip (SoC) for real-time security monitoring by detecting and characterizing different process types, such as file I/O, CPU/ALU-intensive tasks, network I/O and virtualization. We present models that use sensor data for binary classification to determine whether a specific process category is active or inactive. Analyzing the detection results, we determine the importance of each sensor in identifying process types and providing insights into process behaviors and sensor impacts. We developed adaptive ensemble classifiers to accommodate varying load conditions, enhancing detection accuracy across diverse operational scenarios, including regular background activities that simulate real-world conditions. Our results show effective detection of file I/O and CPU/ALU-intensive processes under various loads. Virtualization processes are accurately detected under light loads but show a moderate accuracy decline under heavier conditions. Network I/O detection faces challenges due to fewer relevant sensors. Our ability to predict process categories consistently and with high performance allows us to discover behaviors of various malicious activities. This research underlines the efficacy of sensor-based analysis for reliable and adaptable real-time process monitoring, demonstrating the feasibility of using sensor data for security purposes in various environmental conditions.