Ensemble Classification Model Research Articles

An Ensemble Classification Model to Predict Alzheimer’s Incidence as Multiple Classes

This study introduces an ensemble classification model designed to categorize Alzheimer’s disease (AD) into four distinct classes—mild dementia, no dementia, moderate dementia, and very mild dementia—using Magnetic Resonance Imaging (MRI). The proposed model entitled the Ensemble Classification Model to Predict Alzheimer's Incidence as Multiple Classes (PAIMC) that integrates a six-dimensional analysis of MR images, encompassing entropies, Fractal Dimensions, Gray Level Run Length Matrix (GLRLM), Gray Level Co-occurrence Matrix (GLCM), morphological features, and Local Binary Patterns. A four-fold multi-label cross-validation approach was employed on a benchmark dataset to evaluate the model's performance. Quantitative analysis reveals that PAIMC consistently achieves superior Decision Accuracy, F-Score, Specificity, Sensitivity Recall, and Precision metrics compared to existing state-of-the-art models. For instance, PAIMC's Decision Accuracy and Precision outperform the second-best model by a notable margin across all folds. The model also demonstrates a significant improvement in Sensitivity Recall and Specificity, reinforcing its efficacy in the multi-class classification of AD stages. A novel data diversity assessment measure was developed and utilized, further confirming the robustness of the PAIMC model. The results underscore the potential of PAIMC as a highly accurate tool for AD classification in clinical settings.

SleepBoost: a multi-level tree-based ensemble model for automatic sleep stage classification.

Neurodegenerative diseases often exhibit a strong link with sleep disruption, highlighting the importance of effective sleep stage monitoring. In this light, automatic sleep stage classification (ASSC) plays a pivotal role, now more streamlined than ever due to the advancements in deep learning (DL). However, the opaque nature of DL models can be a barrier in their clinical adoption, due to trust concerns among medical practitioners. To bridge this gap, we introduce SleepBoost, a transparent multi-level tree-based ensemble model specifically designed for ASSC. Our approach includes a crafted feature engineering block (FEB) that extracts 41 time and frequency domain features, out of which 23 are selected based on their high mutual information score (> 0.23). Uniquely, SleepBoost integrates three fundamental linear models into a cohesive multi-level tree structure, further enhanced by a novel reward-based adaptive weight allocation mechanism. Tested on the Sleep-EDF-20 dataset, SleepBoost demonstrates superior performance with an accuracy of 86.3%, F1-score of 80.9%, and Cohen kappa score of 0.807, outperforming leading DL models in ASSC. An ablation study underscores the critical role of our selective feature extraction in enhancing model accuracy and interpretability, crucial for clinical settings. This innovative approach not only offers a more transparent alternative to traditional DL models but also extends potential implications for monitoring and understanding sleep patterns in the context of neurodegenerative disorders. The open-source availability of SleepBoost's implementation at https://github.com/akibzaman/SleepBoost can further facilitate its accessibility and potential for widespread clinical adoption.

DHFS-ECM: Design of a Dual Heuristic Feature Selection-based Ensemble Classification Model for the Identification of Bamboo Species from Genomic Sequences

Background: Analyzing genomic sequences plays a crucial role in understanding biological diversity and classifying Bamboo species. Existing methods for genomic sequence analysis suffer from limitations such as complexity, low accuracy, and the need for constant reconfiguration in response to evolving genomic datasets Aim: This study addresses these limitations by introducing a novel Dual Heuristic Feature Selection- based Ensemble Classification Model (DHFS-ECM) for the precise identification of Bamboo species from genomic sequences. Methods: The proposed DHFS-ECM method employs a Genetic Algorithm to perform dual heuristic feature selection. This process maximizes inter-class variance, leading to the selection of informative N-gram feature sets. Subsequently, intra-class variance levels are used to create optimal training and validation sets, ensuring comprehensive coverage of class-specific features. The selected features are then processed through an ensemble classification layer, combining multiple stratification models for species-specific categorization. Results: Comparative analysis with state-of-the-art methods demonstrate that DHFS-ECM achieves remarkable improvements in accuracy (9.5%), precision (5.9%), recall (8.5%), and AUC performance (4.5%). Importantly, the model maintains its performance even with an increased number of species classes due to the continuous learning facilitated by the Dual Heuristic Genetic Algorithm Model. Conclusion: DHFS-ECM offers several key advantages, including efficient feature extraction, reduced model complexity, enhanced interpretability, and increased robustness and accuracy through the ensemble classification layer. These attributes make DHFS-ECM a promising tool for real-time clinical applications and a valuable contribution to the field of genomic sequence analysis.

Multiple Disease Detection System

Abstract: In the realm of healthcare, the early detection of multiple diseases presents a formidable challenge but holds immense potential for improving patient outcomes. This paper proposes an integrative approach for the simultaneous detection of four prevalent diseases: heart disease, Parkinson's disease, diabetes, and skin cancer. Leveraging advanced machine learning techniques, our framework encompasses data preprocessing methods for cleaning and normalization, feature selection strategies to extract discriminative features from heterogeneous medical datasets, and ensemble classification models for accurate disease prediction. Real-world datasets encompassing diverse medical conditions are utilized to evaluate the efficacy of the proposed framework. Experimental results demonstrate its superior performance in terms of accuracy, sensitivity, and specificity compared to conventional methods. This framework stands as a testament to the transformative potential of technology in healthcare, facilitating early detection and intervention across multiple diseases, thereby enhancing patient care and quality of life.

Deep ensemble model for skin cancer classification with improved feature set

Deep ensemble model for skin cancer classification with improved feature set

Deep learning and genetic algorithm-based ensemble model for feature selection and classification of breast ultrasound images

Feature extraction and selection are important techniques in the classification of medical images. Extraction of key features and selection of relevant features are the preliminary processes that are essential for identifying the shape of an object or diagnosis of a tumor in images. In this study, we conduct a thorough comparison of deep neural networks' performance. The comparison of the networks infers MobileNet as optimal for feature extraction from medical images, where it has minimal parameters and high validation accuracy. For feature selection, we employ the Genetic Algorithm (GA) because of its proficiency in handling high-dimensional and complex feature space. GA's iterative process aligns well with the unique characteristics of breast ultrasound (BUS) images, which enhances its efficacy in selecting salient features of BUS images. An ensemble model, capitalizing on the collective decision-making capabilities of multiple classifiers based on a weighted voting scheme for classification, is proposed. Empirical evaluations are conducted using two publicly available BUS image datasets, BUSI and UDIAT. The proposed model demonstrates a notable improvement of approximately 4% and 9% in accuracy for the BUSI and UDIAT, respectively. The improved diagnostic accuracy in breast abnormality identification allows for early abnormality diagnosis. This improves treatment outcomes for breast cancer patients and highlights the practical value of the proposed method for improving BUS image categorization.

Hybrid one‐dimensional residual autoencoder and ensemble of gradient boosting for cloud IDS

SummaryThe distributed and decentralized architecture of cloud computing is important for a number of industries, including business, government, entertainment, education, and information technology. It facilitates a wide aspect of information technology, where the computing model is vulnerable to attacks or intrusion. For detecting malicious activities, a novel intrusion detection system (IDS) is required to be developed. In this paper, an enhanced synthetic minority oversampling technique (SMOTE) with a hybrid one‐dimensional residual auto encoder and the Ensemble of Gradient Boosting (1D‐RAE‐EGB) models are proposed for cloud intrusion detection. The proposed IDS resolves the class imbalance problem through enhanced SMOTE. In addition, noise reduction is accomplished with ID‐RAE, which minimizes the data dimension. Finally, the soft voting‐based ensemble classification model is used for attack detection. The ensemble gradient boosting model comprises categorical boosting (CatBoost), extreme gradient boosting (XGBoost), and light gradient boosting machine (LightGBM). The ensemble model is fine‐tuned by reducing the number of parameters under fitting conditions since it is not compulsory to re‐adjust the weight values in a backpropagation process. The proposed IDS is implemented in Python using the NSL‐KDD dataset. The accuracy, precision, recall, f1‐score, false positive rate (FPR), false negative rate (FNR), specificity, and Kappa measure obtained for the proposed IDS are 99.98%, 99.3%, 98.5%, 99.95%, 0.723, 0.46, 99.98%, and 99.97%, respectively.

CatNorth: An Improved Gaia DR3 Quasar Candidate Catalog with Pan-STARRS1 and CatWISE

A complete and pure sample of quasars with accurate redshifts is crucial for quasar studies and cosmology. In this paper, we present CatNorth, an improved Gaia Data Release 3 (Gaia DR3) quasar candidate catalog with more than 1.5 million sources in the 3π sky built with data from Gaia, Pan-STARRS1, and CatWISE2020. The XGBoost algorithm is used to reclassify the original Gaia DR3 quasar candidates as stars, galaxies, and quasars. To construct training/validation data sets for the classification, we carefully built two different master stellar samples in addition to the spectroscopic galaxy and quasar samples. An ensemble classification model is obtained by averaging two XGBoost classifiers trained with different master stellar samples. Using a probability threshold of p QSO_mean > 0.95 in our ensemble classification model and an additional cut on the logarithmic probability density of zero proper motion, we retrieved 1,545,514 reliable quasar candidates from the parent Gaia DR3 quasar candidate catalog. We provide photometric redshifts for all candidates with an ensemble regression model. For a subset of 89,100 candidates, accurate spectroscopic redshifts are estimated with the convolutional neural network from the Gaia BP/RP spectra. The CatNorth catalog has a high purity of ∼90%, while maintaining high completeness, which is an ideal sample to understand the quasar population and its statistical properties. The CatNorth catalog is used as the main source of input catalog for the Large Sky Area Multi-Object Fiber Spectroscopic Telescope phase III quasar survey, which is expected to build a highly complete sample of bright quasars with i < 19.5.

House Type Specification for Housing Development Project Using Machine Learning Techniques: A Study From Bangkok Metropolitan Region, Thailand

Specifying the house type of a housing development project is extremely necessary. However, the determination of a project type nowadays has become a delicate matter, requiring the expertise and knowledge of seasoned project developers. This study aimed to apply four machine learning techniques: Decision Tree, Random Forest, Gradient Boosted Tree and Ensemble Classifier, to analyze the data from 179 housing estate projects collected from market reports of real estate companies in Thailand, with a focus on selecting projects with average monthly sales rates that are higher than the average of all acquired projects. This process resulted in a reduced dataset of 59 projects, including 31 townhouses, 22 single-family houses, and six semi-detached houses. As a result, the Ensemble Classifier model has the highest accuracy of 90.91%. The factors most influential in identifying the type of project are the distances from a main road, sky train station, bus station, hospital, and department store. Single-detached house projects are suitable for locations with high potential. The ideal location should be in proximity to a main road, bus station, department store, and hospital. In addition, townhouse projects are ideal for medium-potential locations that are not near shopping malls, but still require proximity to a hospital, sky train station, or bus station. Ultimately, semi-detached house projects are ideal for medium-potential locations that require proximity to a main road for convenient access to sky train station or public transportation, depending on the specific context.

Correction to: Staked deep ensemble model for intruder behaviour detection and classification in cloud

Correction to: Staked deep ensemble model for intruder behaviour detection and classification in cloud

An Empirical Evaluation of a Novel Ensemble Deep Neural Network Model and Explainable AI for Accurate Segmentation and Classification of Ovarian Tumors Using CT Images.

Ovarian cancer is one of the leading causes of death worldwide among the female population. Early diagnosis is crucial for patient treatment. In this work, our main objective is to accurately detect and classify ovarian cancer. To achieve this, two datasets are considered: CT scan images of patients with cancer and those without, and biomarker (clinical parameters) data from all patients. We propose an ensemble deep neural network model and an ensemble machine learning model for the automatic binary classification of ovarian CT scan images and biomarker data. The proposed model incorporates four convolutional neural network models: VGG16, ResNet 152, Inception V3, and DenseNet 101, with transformers applied for feature extraction. These extracted features are fed into our proposed ensemble multi-layer perceptron model for classification. Preprocessing and CNN tuning techniques such as hyperparameter optimization, data augmentation, and fine-tuning are utilized during model training. Our ensemble model outperforms single classifiers and machine learning algorithms, achieving a mean accuracy of 98.96%, a precision of 97.44%, and an F1-score of 98.7%. We compared these results with those obtained using features extracted by the UNet model, followed by classification with our ensemble model. The transformer demonstrated superior performance in feature extraction over the UNet, with a mean Dice score and mean Jaccard score of 0.98 and 0.97, respectively, and standard deviations of 0.04 and 0.06 for benign tumors and 0.99 and 0.98 with standard deviations of 0.01 for malignant tumors. For the biomarker data, the combination of five machine learning models-KNN, logistic regression, SVM, decision tree, and random forest-resulted in an improved accuracy of 92.8% compared to single classifiers.

A novel ensemble model for detecting fake news

&lt;div&gt;Due the growing proliferation of fake news over the past couple of years, our&lt;/div&gt;&lt;div&gt;objective in this paper is to propose an ensemble model for the automatic classification of article news as being either real or fake. For this purpose, we opt&lt;/div&gt;&lt;div&gt;for a blending technique that combines three models, namely bidirectional long&lt;/div&gt;&lt;div&gt;short-term memory (Bi-LSTM), stochastic gradient descent classifier and ridge&lt;/div&gt;&lt;div&gt;classifier. The implementation of the proposed model (i.e. BI-LSR) on real&lt;/div&gt;&lt;div&gt;world datasets, has shown outstanding results. In fact, it achieved an accuracy&lt;/div&gt;&lt;div&gt;score of 99.16%. Accordingly, this ensemble learning has proven to do perform&lt;/div&gt;&lt;div&gt;better than individual conventional machine learning and deep learning models&lt;/div&gt;&lt;div&gt;as well as many ensemble learning approaches cited in the literature.&lt;/div&gt;

An ensemble deep learning model for automatic classification of cotton leaves diseases

&lt;p&gt;Cotton plant (Gossypium herbaceum), is one of the significant fiber crop grown worldwide. However, the crop is quite prone to leaves diseases, for which deep learning (DL) techniques can be utilized for early disease prediction and prevent stakeholders from losing the harvest. The objective of this paper is to develop a novel ensemble based deep convolutional neural network (DCNN) model developed on two base pretrained models named: VGG16 and InceptionV3 for early detection of cotton leaves diseases. The proposed ensemble model trained on cotton leaves dataset reports higher training and testing prediction accuracies as compared to the base pretrained models. Given that, deep learning architectures have hyper-parameters, this paper presents exhaustive experimental evaluations on ensemble model to tune hyper-parameters named learning rate, optimizer and no of epochs. The suggested hyper-parameter settings can be directly utilized while employing the ensemble model for cotton plant leaves disease detection and prediction. With suggested hyper-parameters settings of learning rate 0.0001, 20 epochs and stochastic gradient descent (SGD) optimizer, ensemble model reported training and testing accuracies of 98% and 95% respectively, which was higher than the training and testing accuracies of VGG16 and InceptionV3 pretrained DCNN models.&lt;/p&gt;

An Ensemble Classification Model for Medical Databases Using Hybrid Weights

An Ensemble Classification Model for Medical Databases Using Hybrid Weights

DDoS attack detection in cloud using ensemble model tuned with optimal hyperparameter

SummaryDDoS attacks are a type of cloud incursion that lessen service degradation. DDoS attacks target the cloud network with invalid requests, rejecting legitimate requests. Such attacks disrupt the entire cloud architecture, thus it needs efficient detection methods to spot their presence. This study proposes a novel ensemble classification model for DDoS incursion detection. Pre‐processing, feature extraction, and attack detection are the three main components of the suggested intrusion detection system. Improved data imbalance processing is processed during pre‐processing. Features including HOS‐based features enhanced entropy‐based features, correlation features, and raw features are extracted from the pre‐processed data. The generated features are trained using an ensemble model, which integrates classifiers like SVM, RF, NN, LSTM, and DRN, during the attack detection phase. A new hybrid approach known as TUDMA optimally trains the model by setting the ideal weight because DRN provides the final detected outcome. By reducing errors, this ideal training will guarantee an improvement in detection results. The suggested hybrid optimization combines the two techniques, TDO and DMO. The accuracy of the methods FS‐WOA‐DNN, RBF‐PSO, SMA, BRO, SLO, and NMRA was 86.73%, 85.69%, 84.58%, 87.43%, 88.91%, and 89.78%, respectively, while the accuracy of the TUDMA was 95.34% in the 80th learning percentage. Finally, for many measurements, the suggested efficiency is superior to the traditional methods.

ENSEMBLE CLASSIFIER BASED FAKE NEWS IDENTIFICATION IN ONLINE SOCIAL NETWORK

In today’s context, one of the main mediums of news consumption is through social media. It has become a common trend now to produce fake news for speedy propagation and popularity. Thus, it creates a sort of illusion and deception for readers. Amidst the umpty number of researches done, we could say that there is no research that could accurately predict fake news over online. In this study, the ensemble classifier is employed to develop a model for fake news identification in online social networks. This proposed method applies the stacked ensemble classification model, the proposed approach that learns the represented text model and classifies the textual data into real news and fake news. Then, the proposed approach takes the collaborative decision from the classified data that is generated from the multiple base learners using the weight-based ensemble method. The accuracy prediction and performance evaluation of time consumption of detection fake news are consecutively 80 Percentage and 11 ms respectively. Thus, it improves the efficiency and classification accuracy over large-scale social media messages through an efficient sentiment analysis model.

A Fake Profile Detection Model Using Multistage Stacked Ensemble Classification

Fake profile identification on social media platforms is essential for preserving a reliable online community. Previous studies have primarily used conventional classifiers for fake account identification on social networking sites, neglecting feature selection and class balancing to enhance performance. This study introduces a novel multistage stacked ensemble classification model to enhance fake profile detection accuracy, especially in imbalanced datasets. The model comprises three phases: feature selection, base learning, and meta-learning for classification. The novelty of the work lies in utilizing chi-squared feature-class association-based feature selection, combining stacked ensemble and cost-sensitive learning. The research findings indicate that the proposed model significantly enhances fake profile detection efficiency. Employing cost-sensitive learning enhances accuracy on the Facebook, Instagram, and Twitter spam datasets with 95%, 98.20%, and 81% precision, outperforming conventional and advanced classifiers. It is demonstrated that the proposed model has the potential to enhance the security and reliability of online social networks, compared with existing models.

High-performance imbalanced learning ensembles of decision trees for detecting mineralization anomalies from geochemical exploration data

How to effectively detect geochemical anomalies associated with mineralization is a challenging task due to the extreme class-imbalance of geochemical exploration data. To address this challenge, various machine learning techniques have been employed to detect geochemical anomalies associated with mineralization. However, almost all of these machine learning techniques have their limitations when it comes to modeling class-imbalanced geochemical exploration data. To establish efficient robust high-performance models for detecting geochemical anomalies associated mineralization, a case study was carried out in the Helong area, Jilin Province, China. Decision tree (C4.5) classifiers were used as the base or weak classifiers, four imbalanced learning ensemble models, including self-paced ensemble model, under-bagging ensemble model, synthetic minority oversampling technique (SMOTE)-boost ensemble model and random under sampling (RUS)-boost ensemble model were established and compared in the detection of polymetallic mineralization anomalies from the 1:50,000 stream sediment survey data. The output of the base classifier is a vector consisting of the probability that the sample belongs to a mineralization anomaly and the probability that the sample belongs to the background. The performance of each ensemble model was evaluated using the receiver operating characteristic (ROC) curve and the area under the ROC curve (AUC). The ROC curves of the four models are close to the upper left corner of the ROC space. The AUC values of the four models are not lower than 0.9589. The polymetallic mineralization anomalies detected by the four models account for less than 7 % of the entire study area, but contain not less than 93 % of the polymetallic deposits found in the study area. In addition, the polymetallic mineralization anomalies detected by the four models spatially coincide with the regional controlling factors of polymetallic mineralization in the study area. To summarize, the four models have very high performance in detecting polymetallic mineralization anomalies, and the polymetallic mineralization anomaly detection results are consistent with the regional geological and metallogenic characteristics in the study area. Therefore, the imbalanced learning ensemble techniques are powerful tools for the establishment of high-performance ensemble classification models for detecting mineralization anomalies from geochemical exploration data.

Therapeutic Decision Making in Prevascular Mediastinal Tumors Using CT Radiomics and Clinical Features: Upfront Surgery or Pretreatment Needle Biopsy?

The study aimed to develop machine learning (ML) classification models for differentiating patients who needed direct surgery from patients who needed core needle biopsy among patients with prevascular mediastinal tumor (PMT). Patients with PMT who received a contrast-enhanced computed tomography (CECT) scan and initial management for PMT between January 2010 and December 2020 were included in this retrospective study. Fourteen ML algorithms were used to construct candidate classification models via the voting ensemble approach, based on preoperative clinical data and radiomic features extracted from the CECT. The classification accuracy of clinical diagnosis was 86.1%. The first ensemble learning model was built by randomly choosing seven ML models from a set of fourteen ML models and had a classification accuracy of 88.0% (95% CI = 85.8 to 90.3%). The second ensemble learning model was the combination of five ML models, including NeuralNetFastAI, NeuralNetTorch, RandomForest with Entropy, RandomForest with Gini, and XGBoost, and had a classification accuracy of 90.4% (95% CI = 87.9 to 93.0%), which significantly outperformed clinical diagnosis (p < 0.05). Due to the superior performance, the voting ensemble learning clinical-radiomic classification model may be used as a clinical decision support system to facilitate the selection of the initial management of PMT.

SleepEGAN: A GAN-enhanced ensemble deep learning model for imbalanced classification of sleep stages

Deep neural networks have played an important role in the automatic classification of sleep stages due to their strong representation and in-model feature transformation abilities. However, class imbalance and individual heterogeneity which typically exist in raw EEG signals of sleep data can significantly affect the classification performance of any machine learning algorithms. To solve these two problems, this paper develops a generative adversarial network (GAN)-powered ensemble deep learning model, named SleepEGAN, for the imbalanced classification of sleep stages. To alleviate class imbalance, we propose a new GAN (called EGAN) architecture adapted to the features of EEG signals for data augmentation. The generated samples for minority classes are used in the training process. In addition, we design a cost-free ensemble learning strategy to reduce the model estimation variance caused by the heterogeneity between the validation and test sets, to enhance the accuracy and robustness of prediction performance. We show that the proposed method improves classification accuracy compared to several existing state-of-the-art methods. The overall classification accuracy and macro F1-score obtained by our SleepEGAN method on three public sleep datasets are: Sleep-EDF-39: 86.8% and 81.9%; Sleep-EDF-153: 83.8% and 78.7%; SHHS: 88.0% and 82.1%.