Ensemble Classifier Research Articles

Detecting anomalies in blockchain transactions using machine learning classifiers and explainability analysis

As the use of blockchain for digital payments continues to rise, it becomes susceptible to various malicious attacks. Successfully detecting anomalies within blockchain transactions is essential for bolstering trust in digital payments. However, the task of anomaly detection in blockchain transaction data is challenging due to the infrequent occurrence of illicit transactions. Although several studies have been conducted in the field, a limitation persists: the lack of explanations for the model's predictions. This study seeks to overcome this limitation by integrating explainable artificial intelligence (XAI) techniques and anomaly rules into tree-based ensemble classifiers for detecting anomalous Bitcoin transactions. The shapley additive explanation (SHAP) method is employed to measure the contribution of each feature, and it is compatible with ensemble models. Moreover, we present rules for interpreting whether a Bitcoin transaction is anomalous or not. Additionally, we introduce an under-sampling algorithm named XGBCLUS, designed to balance anomalous and non-anomalous transaction data. This algorithm is compared against other commonly used under-sampling and over-sampling techniques. Finally, the outcomes of various tree-based single classifiers are compared with those of stacking and voting ensemble classifiers. Our experimental results demonstrate that: (i) XGBCLUS enhances true positive rate (TPR) and receiver operating characteristic-area under curve (ROC-AUC) scores compared to state-of-the-art under-sampling and over-sampling techniques, and (ii) our proposed ensemble classifiers outperform traditional single tree-based machine learning classifiers in terms of accuracy, TPR, and false positive rate (FPR) scores.

VoteDroid: a new ensemble voting classifier for malware detection based on fine-tuned deep learning models

AbstractIn this work, VoteDroid a novel fine-tuned deep learning models-based ensemble voting classifier has been proposed for detecting malicious behavior in Android applications. To this end, we proposed adopting the random search optimization algorithm for deciding the structure of the models used as voter classifiers in the ensemble classifier. We specified the potential components that can be used in each model and left the random search algorithm taking a decision about the structure of the model including the number of each component that should be used and its location in the structure. This optimization method has been used to build three different deep learning models namely CNN-ANN, pure CNN, and pure ANN. After selecting the best structure for each DL model, the selected three models have been trained and tested using the constructed image dataset. Afterward, we suggested hybridizing the fine-tuned three deep-learning models to form one ensemble voting classifier with two different working modes namely MMR (Malware Minority Rule) and LMR (Label Majority Rule). To our knowledge, this is the first time that an ensemble classifier has been fine-tuned and hybridized in this way for malware detection. The results showed that the proposed models were promising, where the classification accuracy exceeded 97% in all experiments.

Autism spectrum disorder identification using multi‐model deep ensemble classifier with transfer learning

AbstractIdentifying autism spectrum disorder (ASD) symptoms accurately is a challenging task. The traditional subjective diagnostic process of ASD relies on time‐consuming behavioural and psychological observations. In this study, we introduce an ensemble learning‐based classification model using an open‐access database focusing on functional magnetic resonance imaging (fMRI). We propose a novel multi‐model ensemble classifier (MMEC) and multisite ensemble classifier (MSEC) with transfer learning (TL) for ASD classification to improve the prediction accuracy. The MMEC utilizes four base classifiers, Inception V3, ResNet50, MobileNet, and DenseNet to boost the performance of the individual convolutional neural network (CNN) models. The MSEC combined the base classifiers trained from different data sites. We evaluate the two models with ensemble averaging, weighted averaging, and stacking methods. The proposed MMEC with stacking shows the state of art performance compared to MSEC, improving the prediction accuracy by 3.25%. The obtained results have shown an accuracy of 97.82%, 97.82%, and 97.78% for ensemble averaging, weighted averaging, and stacking methods, respectively, on multi‐site datasets. The ensemble classifier MMEC performed better than a single classifier on the multi‐site dataset. The proposed MMEC opens a new paradigm to design a universal ASD classification framework.

Fault diagnosis method of PEMFC system based on ensemble learning

Due to the coupling of multiple physical fields inside proton exchange membrane fuel cell (PEMFC) and the complex and changeable external application scenarios, the faults of PEMFC system greatly affects its engineering application. The paper proposes an ensemble learning-based fault diagnosis method for PEMFC systems, utilizing five distinct algorithms to develop an integrated ensemble classifier for fault detection. The original sample dataset collected from fuel cells undergoes a process of deep feature extraction to obtain a composite feature sample dataset, which is then inputted into an ensemble classifier to establish a fault diagnosis model. The experimental data of four kinds of faults of 80 W fuel cell test bench, EC fuel cell system and 50 W water-cooled fuel cell test bench are analyzed by using this classification model. The results show that the accuracy and generalization of the ensemble learning algorithm for different fault diagnosis of PEMFC system in different application environments are verified.

Authorship Attribution of English Poetry using Sentiment Analysis

We present a basic methodology and share some interesting results from experiments on using sentiment analysis for authorship attribution of poetry. We demonstrate that sentiment analysis can be effectively used to determine the authorship of poetic works given a sufficiently large training corpus. We also share some promising preliminary results from sentiment-analysis-based attribution of non-poetry works. Most results compare well with traditional authorship attribution approaches. Moreover, adding sentiment analysis to a traditional-feature-based ensemble classifier improved the accuracy of attribution. The strengths and limitations of our methodology and directions for further research are outlined at the end of the paper.

Enhancing EV charging predictions: a comprehensive analysis using K-nearest neighbours and ensemble stack generalization

Enhancing EV charging predictions: a comprehensive analysis using K-nearest neighbours and ensemble stack generalization

A novel code representation for detecting Java code clones using high-level and abstract compiled code representations.

In software development, it's common to reuse existing source code by copying and pasting, resulting in the proliferation of numerous code clones-similar or identical code fragments-that detrimentally affect software quality and maintainability. Although several techniques for code clone detection exist, many encounter challenges in effectively identifying semantic clones due to their inability to extract syntax and semantics information. Fewer techniques leverage low-level source code representations like bytecode or assembly for clone detection. This work introduces a novel code representation for identifying syntactic and semantic clones in Java source code. It integrates high-level features extracted from the Abstract Syntax Tree with low-level features derived from intermediate representations generated by static analysis tools, like the Soot framework. Leveraging this combined representation, fifteen machine-learning models are trained to effectively detect code clones. Evaluation on a large dataset demonstrates the models' efficacy in accurately identifying semantic clones. Among these classifiers, ensemble classifiers, such as the LightGBM classifier, exhibit exceptional accuracy. Linearly combining features enhances the effectiveness of the models compared to multiplication and distance combination techniques. The experimental findings indicate that the proposed method can outperform the current clone detection techniques in detecting semantic clones.

Utilizing Machine Learning Ensemble Techniques for Crime Hotspot Analysis and Prediction

By combining the predictions of trained classifiers, the ensemble learning approach generates new examples through cooperative decision-making. Evidence from early analysis has demonstrated the empirical and logical superiority of ensemble classifiers over single component classifiers. It is still difficult to identify the right configuration for a given dataset, even with the presentation of many ensemble approaches. Many theories based on prediction have been developed to address the topic of machine learning crime prediction in India. The dynamic character of crimes becomes difficult to ascertain. Crime prediction aims to lower crime rates and discourage criminal action. This study provides an authentic and efficient way for determining acceptable crime predictions: the assemble-stacking based crime prediction method (SBCPM), which applies learning-based strategies to produce domain-specific configurations compared to another machine learning model. The result implies that performer models are generally not particularly successful. The ensemble model occasionally outperforms the others with the best correlation coefficient, the lowest average, and the lowest absolute errors. The proposed method generated accurate categorization on the test data. Compared to previous research that just employed violence-based crime records as a baseline, the model's prediction effect is demonstrated to be stronger. The results further shown that criminological theories are congruent with any real-world crime data. The recommended strategy also proved useful in predicting possible crimes. and show that the ensemble model has higher prediction accuracy when compared to the individual classifier.

HSLE: A Hybrid Ensemble Classifier for Prediction of Heart Disease

Background: Detecting heart disease in a timely manner is vital for preventing its progression, as it is the primary cause of death across the globe. Machine learning has the potential to enhance diagnostic accuracy and enable better clinical decision-making. A machine learningpowered hybrid system for diagnosing heart disease may provide a better optimal solution for heart disease prediction. Objective: The overarching objectives include accuracy improvement, enhanced classification reliability, and the development of high-performance prediction models for heart disease. These objectives indicate a commitment to advancing methodologies and models in the field of machine learning and data science, particularly within the domain of healthcare and disease prediction. Method: The proposed system was developed using the Cleveland dataset that was preprocessed and analyzed using Recursive Feature Elimination with Cross-Validation (RFECV) and Least Absolute Shrinkage and Selection Operator (LASSO) feature extraction techniques. Further, a hybrid feature selection approach using RFECV and K-Best has been proposed for feature selection. Eight machine learning classifiers such as Multilayer Perceptron (MLP), Random Forest (RF), Extreme Gradient Boosting (XGB), K-Nearest Neighbours (KNN), Extra Tree (ET), Support Vector Machine (SVC), Adaboost, Decision Tree (DT) were utilized, and the performance of the system was measured in terms of various metrics. Result: The results showed that the proposed HSLE algorithm with hybrid feature selection led to the highest overall accuracy of 98.76%. Conclusion: As mentioned, the main cause of adult death worldwide is chronic disease. Early detection can stop the condition from getting worse. Our research presents an innovative hybrid machine-learning approach designed to forecast heart disease.

Tuberculosis Detection and Air Purifier Suggestion

Tuberculosis (TB) is a disease that can be fatal if not promptly treated. Ensemble deep learning methods have shown promise in aiding the early detection of TB. Previous research typically trained ensemble classifiers using images with similar features, but for optimal performance, an ensemble requires a range of errors, achievable through diverse classification techniques or feature sets. This study focuses on the latter approach, presenting TB detection using deep learning alongside contrast-enhanced canny edge detected (CEED-Canny) X-ray images. CEED-Canny was employed to generate edge-detected lung X-ray images. Two sets of features were derived: one from the enhanced X- ray images and the other from the edge-detected images. By introducing this variation in features, the diversity of errors among the base classifiers was increased, resulting in improved TB detection. The proposed ensemble method achieved a comparable accuracy of 93.59%, sensitivity of 92.31%, and specificity of 94.87% compared to prior research.

An Artificial Intelligence Driven Approach for Classification of Ophthalmic Images using Convolutional Neural Network: An Experimental Study.

Early disease detection is emphasized within ophthalmology now more than ever, and as a result, clinicians and innovators turn to deep learning to expedite accurate diagnosis and mitigate treatment delay. Efforts concentrate on the creation of deep learning systems that analyze clinical image data to detect disease-specific features with maximum sensitivity. Moreover, these systems hold promise of early accurate diagnosis and treatment of patients with common progressive diseases. DenseNet, ResNet, and VGG-16 are among a few of the deep learning Convolutional Neural Network (CNN) algorithms that have been introduced and are being investigated for potential application within ophthalmology. In this study, the authors sought to create and evaluate a novel ensembled deep learning CNN model that analyzes a dataset of shuffled retinal color fundus images (RCFIs) from eyes with various ocular disease features (cataract, glaucoma, diabetic retinopathy). Our aim was to determine (1) the relative performance of our finalized model in classifying RCFIs according to disease and (2) the diagnostic potential of the finalized model to serve as a screening test for specific diseases (cataract, glaucoma, diabetic retinopathy) upon presentation of RCFIs with diverse disease manifestations. We found adding convolutional layers to an existing VGG-16 model, which was named as a proposed model in this article that, resulted in significantly increased performance with 98% accuracy (p<0.05), including good diagnostic potential for binary disease detection in cataract, glaucoma, diabetic retinopathy. The proposed model was found to be suitable and accurate for a decision support system in Ophthalmology Clinical Framework.

DEC-DRR: deep ensemble of classification model for diabetic retinopathy recognition.

Most diabetes patients are liable to have diabetic retinopathy (DR); however, the majority of them might not be even aware of the ailment. Therefore, early detection and treatment of DR are necessary to prevent vision loss. But, avoiding DR is not a simple process. An ophthalmologist can typically identify DR through an optical evaluation of the fundus and through the evaluation of color pictures. However, due to the increased count of DR patients, this could not be possible as it consumes more time. To rectify this problem, a novel deep ensemble-based DR classification technique is developed in this work. Initially, a Wiener filter (WF) is applied for preprocessing the image. Then, the enhanced U-Net-based segmentation process is done. Subsequent to the segmentation process, features are extracted that include statistical features, inferior superior nasal temporal (ISNT), cup to disc ratio (CDR), and improved LGBP as well. Further, deep ensemble classifiers (DEC) like CNN, Bi-GRU, and DMN are used to recognize the disease. The outcomes from DMN, CNN, and Bi-GRU are then subjected to improved SLF. Additionally, the weights of DMN, CNN, and Bi-GRU are adjusted via pelican updated Tasmanian devil optimization (PU-TDO). Finally, outputs on DR (microaneurysms, hemorrhages, hard exudates, and soft exudates) are obtained. The performance of DEC + PU-TDO for diabetic retinopathy is computed over extant models with regard to different measures for four datasets. The results on accuracy using the DEC + PU-TDO scheme for the IDRID dataset is maximum around 0.975 at 90th LP while other models have less accuracy. The FPR of DEC + PU-TDO is less around 0.039 at the 90th LP for the SUSTech-SYSU dataset, while other extant models have maximum FPR.

Enhanced Myocardial Infarction Prediction Using Machine Learning Algorithms and Gender-Specific Insights

A serious medical condition called myocardial infarction (MI), sometimes referred to as a "heart attack," is caused by disruptions in the blood supply to the myocardium. This research examines the efficacy of machine learning (ML) algorithms in forecasting myocardial infarction (MI) using a dataset of 350 records. The study identifies key risk factors for predicting myocardial infarction (MI), such as elevated cholesterol levels, diabetes, advanced age, overall health status, mental well-being, obesity, physical activity, smoking habits, hypertension, and depression. Significantly, gender does not manifest as a predictor of myocardial infarction (MI) when employing various classification methods. The research achieves high accuracy rates of 89.32%, 87.53%, 81.29%, and 76.59% using different machine learning algorithms, including Deep Belief Network (DBN), C4.5, Random Forest (RF), and Bayesian Network (BN), respectively. Algorithm-specific rule sets identify correlations, with the C4.5 algorithm revealing interesting connections between smoking habits and protection against myocardial infarction (MI). Performance metrics like accuracy, precision, sensitivity, and specificity attest to the effectiveness of the proposed technique. The results demonstrate the superior performance of the DBN algorithm, surpassing other algorithms in terms of accuracy (89.32%), precision (84.04%), sensitivity (86.63%), and specificity (82.45%). This paper provides crucial insights into predictive modeling for myocardial infarction (MI), highlighting the importance of various risk factors and advanced machine learning (ML) algorithms. The results offer clinicians and researchers a strong foundation for comprehending and potentially averting myocardial infarction, relying on personalized patient profiles. This paper has the potential to significantly contribute to the field by applying ensemble classifiers and machine learning models to forecast gender-specific myocardial infarctions. As a result, diagnostic precision and patient outcomes could be revolutionized.

A hybrid quantum ensemble learning model for malicious code detection

Quantum computing as a new computing model with parallel computing capability and high information carrying capacity, has attracted a lot of attention from researchers. Ensemble learning is an effective strategy often used in machine learning to improve the performance of weak classifiers. Currently, the classification performance of quantum classifiers is not satisfactory enough due to factors such as the depth of quantum circuit, quantum noise, and quantum coding method, etc. For this reason, this paper combined the ensemble learning idea and quantum classifiers to design a novel hybrid quantum machine learning model. Firstly, we run the Stacking method in classical machine learning to realize the dimensionality reduction of high-latitude data while ensuring the validity of data features. Secondly, we used the Bagging method and Bayesian hyperparameter optimization method applied to quantum support vector machine (QSVM), quantum K nearest neighbors (QKNN), variational quantum classifier (VQC). Thirdly, the voting method is used to ensemble the predict results of QSVM, QKNN, VQC as the final result. We applied the hybrid quantum ensemble machine learning model to malicious code detection. The experimental results show that the classification precision (accuracy, F1-score) of this model has been improved to 98.9% (94.5%, 94.24%). Combined with the acceleration of quantum computing and the higher precision rate, it can effectively deal with the growing trend of malicious codes, which is of great significance to cyberspace security.

Enhanced Diabetic Retinopathy Diagnosis: A Comparative Analysis of models with an Ensemble classifier and Deep Q Learning

- A novel method that combines the strengths of different classifiers such as Naive Bayes, Multi-Layer Perceptron (MLP), and Support Vector Machine (SVM) is introduced in this paper. This method tackles the urgent need for cutting-edge diagnostic methods in the field of ophthalmology, mainly for the identification of diabetic retinopathy (DR). The approach is ensemble-based. Classical methods in retinal analysis of images often fail as they are static and are unable to adjust to the unique details that each distinct image presents. This constraint results in less accurate and precise diagnostic results, highlighting the urgent need for more adaptable and dynamic methods. The suggested model differs significantly from previous methods. Through the use of an ensemble approach, it capitalizes on the distinct advantages of each classifier: the MLP process's sophisticated feature extraction skill, Naive Bayes' probabilistic analysis, and SVM's non-linear pattern recognition capacity. By combining these techniques, the inherent drawbacks of utilizing a single strategy are addressed, guaranteeing a more thorough examination of retinal samples and images. The core of this idea is the system using Deep Q Learning (DQL) for adaptive classifier selection. Using learned Q Values for various contexts, this reinforcement learning technique selects the best classifier adaptively for each unique retinal image, hence optimizing the ensemble. This approach not only advances diagnosis accuracy and precision but also guarantees ongoing learning and adaptation to keep up with changing data patterns and advances in imaging technology. Extensive experiments on the IDRiD &amp; EyePACS Dataset show the effectiveness of this model with a 5.5% increase in overall accuracy with other performance metrics, the results show a significant improvement over the current method. They represent a significant advancement in the timely and accurate identification of diabetic retinopathy, which will ultimately benefit patients and lessen the strain on healthcare systems. Thus, this work represents a major step forward in patient care as well as a technological advance, opening the door to more efficient supervision and treatment of retinal illnesses.

Enhancements in Immediate Speech Emotion Detection: Harnessing Prosodic and Spectral Characteristics

Speech is essential to human communication for expressing and understanding feelings. Emotional speech processing has challenges with expert data sampling, dataset organization, and computational complexity in large-scale analysis. This study aims to reduce data redundancy and high dimensionality by introducing a new speech emotion recognition system. The system employs Diffusion Map to reduce dimensionality and includes Decision Trees and K-Nearest Neighbors(KNN)ensemble classifiers. These strategies are suggested to increase voice emotion recognition accuracy. Speech emotion recognition is gaining popularity in affective computing for usage in medical, industry, and academics. This project aims to provide an efficient and robust real-time emotion identification framework. In order to identify emotions using supervised machine learning models, this work makes use of paralinguistic factors such as intensity, pitch, and MFCC. In order to classify data, experimental analysis integrates prosodic and spectral information utilizing methods like Random Forest, Multilayer Perceptron, SVM, KNN, and Gaussian Naïve Bayes. Fast training times make these machine learning models excellent for real-time applications. SVM and MLP have the highest accuracy at 70.86% and 79.52%, respectively. Comparisons to benchmarks show significant improvements over earlier models.

A novel multi-sensor hybrid fusion framework

Multi-sensor data fusion has emerged as a powerful approach to enhance the accuracy and robustness of diagnostic systems. However, effectively integrating multiple sensor data remains a challenge. To address this issue, this paper proposes a novel multi-sensor fusion framework. Firstly, a vibration signal weighted fusion rule based on Kullback–Leibler divergence-permutation entropy is introduced, which adaptively determines the weighting coefficients by considering the positional differences of different sensors. Secondly, a lightweight multi-scale convolutional neural network is designed for feature extraction and fusion of multi-sensor data. An ensemble classifier is employed for fault classification, and an improved hard voting strategy is proposed to achieve more reliable decision fusion. Finally, the superiority of the proposed method is validated using modular state detection data from the Kaggle database.

Ensemble Learning for Higher Diagnostic Precision in Schizophrenia Using Peripheral Blood Gene Expression Profile.

Stigma contributes to a significant part of the burden of schizophrenia (SCZ), therefore reducing false positives from the diagnosis would be liberating for the individuals with SCZ and desirable for the clinicians. The stigmatization associated with schizophrenia advocates the need for high-precision diagnosis. In this study, we present an ensemble learning-based approach for high-precision diagnosis of SCZ using peripheral blood gene expression profiles. The machine learning (ML) models, support vector machines (SVM), and prediction analysis for microarrays (PAM) were developed using differentially expressed genes (DEGs) as features. The SCZ samples were classified based on a voting ensemble classifier of SVM and PAM. Further, microarray-based learning was used to classify RNA sequencing (RNA-Seq) samples from our case-control study (Pune-SCZ) to assess cross-platform compatibility. Ensemble learning using ML models resulted in a significantly higher precision of 80.41% (SD: 0.04) when compared to the individual models (SVM-radial: 71.69%, SD: 0.04 and PAM 77.20%, SD: 0.02). The RNA sequencing samples from our case-control study (Pune-SCZ) resulted in a moderate precision (59.92%, SD: 0.05). The feature genes used for model building were enriched for biological processes such as response to stress, regulation of the immune system, and metabolism of organic nitrogen compounds. The network analysis identified RBX1, CUL4B, DDB1, PRPF19, and COPS4 as hub genes. In summary, this study developed robust models for higher diagnostic precision in psychiatric disorders. Future efforts will be directed towards multi-omic integration and developing "explainable" diagnostic models.

SAR Image Fusion Classification Based on Improved D-S Evidence Theory

Applications for classifying Synthetic Aperture Radar (SAR) images are critical to environmental monitoring, urban planning, and land resource surveying. Fusion approaches work well for increasing SAR image categorization accuracy. However, effective processing of uncertainty information is frequently overlooked by conventional fusion classification systems. This paper presented a fusion classification approach for SAR images based on improved Dempster-Shafer (D-S) evidence theory, taking into account the uncertainty and possible conflicts in the classifiers’ output. First, after adaptively identifying the optimal hyperparameters of models, the SAR data is identified using Gradient Boosting Machine (GBM), Multilayer Perceptron (MLP), and Random Forest (RF). Different weights are then allocated in accordance with the variations in classifier performance. Ultimately, enhanced D-S evidence theory is used to integrate the output of these classifiers. In addition, uncertainty in classification results is also modeled and visualized. Experiments with Flevoland and Oberpfaffenhofen datasets show that the accuracy of this method is 85.21% and 91.88% respectively, outperforming both the soft voting ensemble classifier (SVE) and D-S evidence theory fusion method. This approach enhances the comprehension and capacity to manage uncertainty in the model output in addition to increasing the accuracy and reliability of the classification of SAR data.

Reducing the volume of computations when building analogs of neural networks for the first stage of an ensemble classifier with stacking

The object of research in this work is ensemble classifiers with stacking, intended for the classification of objects in images with the presence of small sets of labeled data for training. To improve the quality of classification at the first stage of such a classifier, it is necessary to place more primary classifiers that differ in heterogeneous structured processing. However, the number of known neural networks with appropriate characteristics is limited. One approach to solving this problem is to build analogs of known neural networks that make classification errors on other images compared to the base network. The disadvantage of the known methods for constructing such analogs is the need to perform additional floating-point operations. The current paper proposes and investigates a new method to form analogs through random cyclic shifts of rows or columns of input images. This has made it possible to completely eliminate additional floating-point operations. The effectiveness of using this method is explained by the structured processing of input images in basic neural networks. The use of analogs obtained by the proposed method does not impose additional restrictions in practice. This is because the heterogeneity of structured processing in basic neural networks is a typical requirement for them in an ensemble classifier with stacking. The simulation for the CIFAR-10 data set demonstrated that the proposed technique for constructing analogs allows for a comparative quality of classification by the ensemble classifier. Using MLP-Mixer analogs provided an improvement of 4.6 %, and CCT analogs – 5.9 %