Voting-based Ensemble Research Articles

A novel weighted majority voting-based ensemble approach for detection of road accidents using social media data

A novel weighted majority voting-based ensemble approach for detection of road accidents using social media data

Predicting Mental Health Outcomes Using Wearable Device Data and Machine Learning

This paper proposes a machine learning- based system designed to predict mental health outcomes using wearable device data. The system is conceptualized to process physiological and behavioral data such as heart rate, sleep patterns, and activity levels collected from wearable technology. Key stages of the system include data preprocessing, feature extraction, and model training using multiple machine-learning algorithms, including Random Forest, Support Vector Machine, XGBoost, and Logistic Regression. These models are combined using a voting-based ensemble classifier to improve prediction accuracy. While the system has not yet been implemented, expected results suggest that this approach will enhance prediction reliability and offer real-time insights into mental health conditions. The proposed system is envisioned to facilitate early detection of mental health disorders, thereby aiding in timely interventions and personalized care.

Brake fault diagnosis using a voting ensemble of machine learning classifiers

Brake fault diagnosis is a crucial aspect of enhancing driving safety, as various faults such as air in brake fluid, brake oil spill, reservoir leak, mechanical fade and distinct types of brake pad wear can compromise vehicle safety. This study presents a method for timely fault detection by analyzing vibration signals. Vibration signals for normal and faulty conditions were captured using a hydraulic brake test setup equipped with a piezoelectric transducer and data acquisition system. Feature extraction was performed using an auto-regressive moving average (ARMA) model. The performance of five different classifiers namely, random forest (RF), Naive Bayes (NB), instance-based k-nearest neighbours (IBk), logistic model trees (LMT) and J48 decision tree was evaluated. The LMT classifier achieved the highest accuracy at 95.00 % followed by IBk, RF, J48 and NB with accuracies of 92.00 %, 90.00 %, 90.00 % and 87.00 %. To further improve the diagnosis accuracy, a voting-based ensemble approach was employed by combining two, three, four and five classifiers with the application of five different voting strategies. The results obtained showcase that a combination of three classifiers LMT, IBk and NB utilizing the majority voting rule yielded an enhanced classification accuracy of 98.00 % highlighting the effectiveness of this ensemble method in brake fault diagnosis. A similar classification accuracy of 98.00 % was achieved by four (LMT-IBk-RF-J48) and five (IBk-RF-MLP-J48-NB) classifier ensembles; however, considering the computational complexity three classifier ensembles were selected. Whilst two classifier ensembles (IBk-J48) achieved a maximum classification accuracy of 96.00 %.

Artificial intelligence in the classification and segmentation of fundus images with choroidal nevi

ObjectiveThe purpose of this study is to summarize the results from 3 experimental studies into the use of artificial intelligence to classify and segment colour fundus images with choroidal nevi. Study DesignThis study is based on a secondary analysis of colour fundus images taken of patients receiving usual clinical care from the Alberta Ocular Brachytherapy Program. MethodsHigh-resolution colour fundus images were labeled by experienced ocular oncologists. In experimental study 1, four pre-trained models (ResNet 50, VGG-19, VGG-16, and AlexNet) were evaluated for their ability to classify images based on the presence of choroidal nevi. In experimental study 2, the performance of 3 patch-based models to classify images based on the presence of choroidal nevi were compared. In experimental study 3, four convolutional neural network models were developed to segment the images. In experimental studies 1 and 2, performance was measured using accuracy, precision, recall, F1 score, and AUC. In experimental study 3, IoU and Dice measures were used to evaluate performance. ResultsA total of 591 labelled colour fundus images were used for analysis. In experimental study 1, VGG-16 showed the best accuracy, AUC, and recall, but lower precision in classifying images. In experimental study 2, the patched approached enhanced with artifact and contrast outperformed the others in classifying images. In experimental study 3, a voting-based Ensemble model excelled in segmenting the part of images with nevi. ConclusionsIt is feasible to train AI models to identify choroidal nevi in colour fundus images.

A voting-based ensemble classifier to predict phases and crystal structures of high entropy alloys through thermodynamic, electronic, and configurational parameters

This study aims to predict the various phases present in high entropy alloys (HEAs) and consequently classify their crystal structure employing multiple machine learning (ML) algorithms utilizing five thermodynamic, electronic and configurational parameters which are considered to be essential for the formation of HEA phases. The properties of a high entropy alloy can eventually be traced through accurate phase and crystal structure prediction, which is essential for selecting the ideal elements for designs. Twelve distinct ML algorithms were executed to predict the phases of HEAs, adopting an experimental database of 322 different HEAs, involving 33 amorphous (AM), 31 intermetallics (IM), and 258 solid solutions (SS) phases. Among the twelve ML models, Cat Boost Classifier displayed the optimum accuracy of 98.06 % for phase predictions. Further, crystal structure classification of the SS phase (body-centered cubic- BCC, face-centered cubic- FCC, and mixed body-centered and face-centered cubic- BCC+FCC) has endeavoured for better microstructure evolution using a different database containing of 194 additional HEAs data with 61 FCC, 76 BCC, and 57 BCC+FCC crystal structures and in comparison to the other models tested, the Gradient Boosting Classifier evolved with the highest accuracy of 86.90 %. An ensemble classifier was also introduced to improve the performance of the ML models, resulting in an accuracy increase to 98.70 % and 86.95 % for phase and crystal structure predictions, respectively. Additionally, the influence of parameters on model accuracy was determined independently.

Voting based ensemble for detecting visual faults in photovoltaic modules using AlexNet features

This study proposes a novel approach utilizing a voting-based ensemble technique to diagnose visible faults in photovoltaic (PV) modules from aerial images captured by unmanned aerial vehicles (UAVs), leveraging AlexNet features. The proposed method focuses on classifying commonly occurring visual faults such as glass breakage, snail trails, burn marks, delamination and discoloration. Two voting-based ensemble models, a two-class ensemble (combining support vector machines and k-nearest neighbor) and a three-class ensemble (integrating support vector machines, J48, and k-nearest neighbor) were developed and evaluated against individual machine learning classifiers. Results indicate that the two-class ensemble outperforms the three-class ensemble and other individual classifiers, achieving an accuracy of 98.30%. This approach not only enhances fault diagnosis accuracy but also reduces inspection costs and instrument monitoring efforts contributing to the sustainable and efficient operation of PV systems.

An ensemble approach for electrocardiogram and lip features based biometric authentication by using grey wolf optimization

In the pursuit of fortified security measures, the convergence of multimodal biometric authentication and ensemble learning techniques have emerged as a pivotal domain of research. This study explores the integration of multimodal biometric authentication and ensemble learning techniques to enhance security. Focusing on lip movement and electrocardiogram (ECG) data, the research combines their distinct characteristics for advanced authentication. Ensemble learning merges diverse models, achieving increased accuracy and resilience in multimodal fusion. Harmonizing lip and ECG modalities establishes a robust authentication system, countering vulnerabilities in unimodal methods. This approach leverages ECG's robustness against spoofing attacks and lip's fine-grained behavioral cues for comprehensive authentication. Ensemble learning techniques, from majority voting to advanced methods, harness the strengths of individual models, improving accuracy, reliability, and generalization. Moreover, ensemble learning detects anomalies, enhancing security. The study incorporates ECG signal filtering and lip region extraction as preprocessing, uses wavelet transform for ECG features, SIFT for lip image features, and employs greywolf optimization for feature selection. Ultimately, a voting-based ensemble classifier is applied for classification, showcasing the potential of this integrated approach in fortified security measures.

Effective Voting-Based Ensemble Learning for Segregated Load Forecasting With Low Sampling Data

Effective Voting-Based Ensemble Learning for Segregated Load Forecasting With Low Sampling Data

Using Voting-Based Ensemble Classifiers to Map Invasive Phragmites australis

Machine learning is frequently combined with imagery acquired from uncrewed aircraft systems (UASs) to detect invasive plants. Having prior knowledge of which machine learning algorithm will produce the most accurate results is difficult. This study examines the efficacy of a voting-based ensemble classifier to identify invasive Phragmites australis from three-band (red, green, blue; RGB) and five-band (red, green, blue, red edge, near-infrared; multispectral; MS) UAS imagery acquired over multiple Minnesota wetlands. A Random Forest, histogram-based gradient-boosting classification tree, and two artificial neural networks were used within the voting-based ensemble classifier. Classifications from the RGB and multispectral imagery were compared across validation sites both with and without post-processing from an object-based image analysis (OBIA) workflow (post-machine learning OBIA rule set; post-ML OBIA rule set). Results from this study suggest that a voting-based ensemble classifier can accurately identify invasive Phragmites australis from RGB and multispectral imagery. Accuracies greater than 80% were attained by the voting-based ensemble classifier for both the RGB and multispectral imagery. The highest accuracy, 91%, was achieved when using the multispectral imagery, a canopy height model, and a post-ML OBIA rule set. The study emphasizes the need for further research regarding the accurate identification of Phragmites australis at low stem densities.

Detection of Algorithmically Generated Domain Names using Ensemble Machine Learning Technique

Prior to now, cyber attackers use malwares with hard-coded domain names stored in the malware binaries that communicate with a command and control (C&C) servers to launch cyber-attacks on their victim computers. Malware attacks such as botnets and ransomwares are some of the most prevalent forms of these attacks. As soon as a system is infected with a malware (either a botnet or a ransomware), one of the most essential components is to establish a secured communication with the botmaster (i.e., the malware author), through a C&C server. However, with a simple reverse engineering technique, cyber security experts could detect and block these domain names, hence, denying them the ability to communicate with the C&C servers and from receiving further instructions from the botmaster. This led to cyber criminals developing the Domain Generation Algorithm (DGA) technique, which algorithmically generate thousands or more candidate’s domain names for communication with the C&C server, thereby obfuscating the domain names of these malwares and making it difficult for cyber security experts to detect or block these domain names. This paper therefore proposes an ensemble machine learning technique for the detection and classification of algorithmically generated domain names (AGDNs) leveraging the combined strength of 4 different machine learning algorithms: Naïve Bayes, SVM, Random Forest and CART. The models were trained twice, first with 4 features and thereafter with 10 features. In order to effectively utilise the result of the predictions, we used a voting-based ensemble approach, where the final classification is decided by the majority vote of the algorithms. Result of the research shows that the Naïve Bayes model performed better than all the other models with an accuracy of 97.54% when trained with 10 features and 95.99% when trained with 4 features. Keywords: WSN, DDoS, Intrusion Detection System, Random Forest, Machine Learning. Proceedings Citation Format Abdullahi, S.M., Mohammed, A., Ibrahim, R.Y. & Shamsuddeen, A. (2023): Detection of Algorithmically Generated Domain Names using Ensemble Machine Learning Technique. Proceedings of the Cyber Secure Nigeria Conference. Nigerian Army Resource Centre (NARC) Abuja, Nigeria. 11-12th July, 2023. Pp 27-34. https://cybersecurenigeria.org/conference-proceedings/volume-2-2023/ dx.doi.org/10.22624/AIMS/CSEAN-SMART2023P2.

EL V.2 Model for Predicting Food Safety Risks at Taiwan Border Using the Voting-Based Ensemble Method.

Border management serves as a crucial control checkpoint for governments to regulate the quality and safety of imported food. In 2020, the first-generation ensemble learning prediction model (EL V.1) was introduced to Taiwan's border food management. This model primarily assesses the risk of imported food by combining five algorithms to determine whether quality sampling should be performed on imported food at the border. In this study, a second-generation ensemble learning prediction model (EL V.2) was developed based on seven algorithms to enhance the "detection rate of unqualified cases" and improve the robustness of the model. In this study, Elastic Net was used to select the characteristic risk factors. Two algorithms were used to construct the new model: The Bagging-Gradient Boosting Machine and Bagging-Elastic Net. In addition, Fβ was used to flexibly control the sampling rate, improving the predictive performance and robustness of the model. The chi-square test was employed to compare the efficacy of "pre-launch (2019) random sampling inspection" and "post-launch (2020-2022) model prediction sampling inspection". For cases recommended for inspection by the ensemble learning model and subsequently inspected, the unqualified rates were 5.10%, 6.36%, and 4.39% in 2020, 2021, and 2022, respectively, which were significantly higher (p < 0.001) compared with the random sampling rate of 2.09% in 2019. The prediction indices established by the confusion matrix were used to further evaluate the prediction effects of EL V.1 and EL V.2, and the EL V.2 model exhibited superior predictive performance compared with EL V.1, and both models outperformed random sampling.

Early Detection of Alzheimer's Disease Using Cognitive Features: A Voting-Based Ensemble Machine Learning Approach

Early detection of Alzheimer's disease (AD) is vital for adequate control. Machine learning techniques have gained much attraction due to their efficiency in predicting AD using cognitive tests. Ensemble machine learning models are helpful in improving the robustness of the learning system via combining multiple machine learning models. This article proposes a novel ensemble machine learning technique for the early detection of AD. First, a novel feature selection technique referred to as Neighborhood Component Analysis and Correlation-based Filtration (NCA-F) is proposed to select the vital cognitive features from a given dataset. Second, various machine learning classifiers were trained using the proposed NCA-F method. The top classifiers were selected for voting based on the performance results. The voting is performed using an adaptive weight matrix process. The output label of a model is multiplied by the F1 score and represented as weight. The results revealed an accuracy of 93.92% when using adaptive voting, which is better than the accuracy of 90.53% observed when using the traditional artificial neural network method. The proposed technique improved the accuracy of detecting AD at an early stage. Furthermore, the results against a recent study using same features also revealed an improvement of 12.12% in accuracy.

Hybrid ensemble framework with self-attention mechanism for social spam detection on imbalanced data

Cybercriminals use social media platforms to disseminate spam, misleading facts, fake news, and malicious links. Blocking such deceptive social media spam is essential. However, extracting relevant features from social networks is challenging due to privacy and time constraints. Traditional frequency-based word representation techniques are time-consuming and inefficient in producing contextual word vectors. Word embeddings and deep learning models have recently shown good results in text classification. Also, most existing approaches assumed balanced class distribution, which is false for most real-world datasets. In this paper, an attempt is made to advance the performance of the social spam detection system by leveraging dataset balancing, advanced word embedding techniques, machine learning, and deep learning approaches with the self-attention mechanism. In the proposed framework, the datasets are balanced using NearMiss and SmoteTomek techniques to feed several machine-learning models. Later, the baseline ML models and proposed voting-based ensemble models are evaluated on imbalanced and balanced datasets. For the proposed deep learning-based hybrid approaches, embeddings are generated using GloVe and FastText word embeddings on the balanced combined dataset and passed into the deep neural network comprised of Conv1D and Bi-directional recurrent neural network layers with the self-attention mechanism for improved context understanding and effective results. This study examines hybrid approaches for detecting social spam using imbalanced social network data and picks the optimum combination. Besides, Machine learning ensembles, word embeddings, deep learning with hyper-parameter optimization, and a self-attention method are compared thoroughly. Experiments and comparisons with other techniques show that the proposed hybrid framework with deep learning-based approaches achieves better performance.

Voting-based Ensemble Learning approach for Cyber Attacks Detection in Industrial Internet of Things

The Industrial Internet of Things (IIoT) is a fast developing interconnection of smart technologies that have revolutionised the way the industries work by combining sensors and devices by automating their regular operations through Internet of Things (IoT). In various application domains, IoT devices provide seamless connectivity and diversity. Because of their round-the-clock connectivity these systems and their communication channel are subjected to targeted cyber-attacks. As a result, a multi-level security solution is required to protect the industrial system. An Intrusion Detection System (IDS) can be used to counteract these cyber-attacks by analyzing the data packets for any targeted attacks in the IIoT environment. This paper proposes a cyber-attacks detection framework for IIoT by using the Voting-based Ensemble Learning approach. In the proposed framework, an ensemble of the latest and classical Machine Learning (ML) techniques like Histogram Gradient Boosting (HGB), CatBoost, and Random Forest (RF), and a hard voting classifier are employed for efficient detection of cyber-attacks. In our proposed model, CatBoost exceeds the other two with an accuracy of 99.85 %, while HGB and RF have an accuracy of 97.90 % and 98.83 % respectively.

An automated system for the assessment and grading of adolescent delinquency using a machine learning-based soft voting framework

Adolescent (or juvenile) delinquency is defined as the habitual engagement in unlawful behavior of a minor under the age of majority. According to studies, the likelihood of acquiring a deviant personality increases significantly during adolescence. As a result, identifying deviant youth early and providing proper medical counseling makes perfect sense. Due to the scarcity of qualified clinicians, human appraisal of individual adolescent behavior is subjective and time-consuming. As a result, a machine learning-based intelligent automated system for assessing and grading delinquency levels in teenagers at an early stage must be devised. To solve this problem, a soft voting-based ensemble classification model has been developed that includes a Decision Tree, Multi-layer Perceptron, and Support Vector Machine as base classifiers to accurately classify teenagers into three groups based on severity levels, viz., low, medium, and high. Over the normalized structured behavioral data, the proposed soft voting-based model outperforms all other individual classifiers with 87.50% accuracy, an AUC of 0.94, 0.81 Kappa value, and an F-score of 0.88.

Voting Combinations-Based Ensemble: A Hybrid Approach

Machine learning (ML) is a prominent and extensively researched field in the artificial intelligence area which assists to strengthen the accomplishment of classification. In this study, the main idea is to provide the classification and analysis of ML and Ensemble Learning (EL) algorithms. To support this idea, six supervised ML algorithms, C4.5 (J48), K-Nearest Neighbor (KNN), Logistic Regression (LR), Support Vector Machine (SVM), Naive Bayes (NB) and One Rule (OneR) in addition the five UCI Datasets of ML Repository, are being applied that demonstrates the robustness and effectiveness of numerous approaches. In this paper, a voting-based ensemble classifier has been proposed along with two base learners (namely, Random Forest and Rotation Forest) to progress the performance. Whereas, for analytical procedures, significant parameters have been considered: Accuracy, Area under Curve (AUC), recall, precision, and F-measure values. Hence, the prime objective of this research is to obtain binary classification and efficiency by conducting the progress of ML and EL approaches. We present experimental outcomes that validate the effectiveness of our method to well-known competitive approaches. Image recognition and ML challenges, such as binary classification, can be solved using this method.

An Empirical Study of the Effectiveness of an Ensemble of Stand-alone Sentiment Detection Tools for Software Engineering Datasets

Sentiment analysis in software engineering (SE) has shown promise to analyze and support diverse development activities. Recently, several tools are proposed to detect sentiments in software artifacts. While the tools improve accuracy over off-the-shelf tools, recent research shows that their performance could still be unsatisfactory. A more accurate sentiment detector for SE can help reduce noise in analysis of software scenarios where sentiment analysis is required. Recently, combinations, i.e., hybrids of stand-alone classifiers are found to offer better performance than the stand-alone classifiers for fault detection. However, we are aware of no such approach for sentiment detection for software artifacts. We report the results of an empirical study that we conducted to determine the feasibility of developing an ensemble engine by combining the polarity labels of stand-alone SE-specific sentiment detectors. Our study has two phases. In the first phase, we pick five SE-specific sentiment detection tools from two recently published papers by Lin et al. [ 29 , 30 ], who first reported negative results with stand alone sentiment detectors and then proposed an improved SE-specific sentiment detector, POME [ 29 ]. We report the study results on 17,581 units (sentences/documents) coming from six currently available sentiment benchmarks for software engineering. We find that the existing tools can be complementary to each other in 85-95% of the cases, i.e., one is wrong but another is right. However, a majority voting-based ensemble of those tools fails to improve the accuracy of sentiment detection. We develop Sentisead, a supervised tool by combining the polarity labels and bag of words as features. Sentisead improves the performance (F1-score) of the individual tools by 4% (over Senti4SD [ 5 ]) – 100% (over POME [ 29 ]). The initial development of Sentisead occurred before we observed the use of deep learning models for SE-specific sentiment detection. In particular, recent papers show the superiority of advanced language-based pre-trained transformer models (PTM) over rule-based and shallow learning models. Consequently, in a second phase, we compare and improve Sentisead infrastructure using the PTMs. We find that a Sentisead infrastructure with RoBERTa as the ensemble of the five stand-alone rule-based and shallow learning SE-specific tools from Lin et al. [ 29 , 30 ] offers the best F1-score of 0.805 across the six datasets, while a stand-alone RoBERTa shows an F1-score of 0.801.

Soft Voting-based Ensemble Model for Bengali Sign Gesture Recognition

Human hand gestures are becoming one of the most important, intuitive, and essential means of recognizing sign language. Sign language is used to convey different meanings through visual-manual methods. Hand gestures help the hearing impaired to communicate. Nevertheless, it is very difficult to achieve a high recognition rate of hand gestures due to the environment and physical anatomy of human beings such as light condition, hand size, position, and uncontrolled environment. Moreover, the recognition of appropriate gestures is currently considered a major challenge. In this context, this paper proposes a probabilistic soft voting-based ensemble model to recognize Bengali sign gestures. We have divided this study into pre-processing, data augmentation and ensemble model-based voting process, and classification for gesture recognition. The purpose of pre-processing is to remove noise from input images, resize it, and segment hand gestures. Data augmentation is applied to create a larger database for in-depth model training. Finally, the ensemble model consists of a support vector machine (SVM), random forest (RF), and convolution neural network (CNN) is used to train and classify gestures. Whereas, the ReLu activation function is used in CNN to solve neuron death problems and to accelerate RF classification through principal component analysis (PCA). A Bengali Sign Number Dataset named “BSN-Dataset” is proposed for model performance. The proposed technique enhances sign gesture recognition capabilities by utilizing segmentation, augmentation, and soft-voting classifiers which have obtained an average of 99.50% greater performance than CNN, RF, and SVM individually, as well as significantly more accuracy than existing systems.

A Voting-Based Ensemble Deep Learning Method Focused on Multi-Step Prediction of Food Safety Risk Levels: Applications in Hazard Analysis of Heavy Metals in Grain Processing Products.

Grain processing products constitute an essential component of the human diet and are among the main sources of heavy metal intake. Therefore, a systematic assessment of risk factors and early-warning systems are vital to control heavy metal hazards in grain processing products. In this study, we established a risk assessment model to systematically analyze heavy metal hazards and combined the model with the K-means++ algorithm to perform risk level classification. We then employed deep learning models to conduct a multi-step prediction of risk levels, providing an early warning of food safety risks. By introducing a voting-ensemble technique, the accuracy of the prediction model was improved. The results indicated that the proposed model was superior to other models, exhibiting the overall accuracy of 90.47% in the 7-day prediction and thus satisfying the basic requirement of the food supervision department. This study provides a novel early-warning model for the systematic assessment of the risk level and further allows the development of targeted regulatory strategies to improve supervision efficiency.

Forecasting US dollar exchange rate movement with computational models and human behavior

Our research investigates the potential benefits of adding the Behavioral Finance approach to the Machine Learning and Big Data framework applied to the challenging problem of forecasting the US Dollar exchange rate. More specifically, we show how to improve existing voting-based ensemble models trained to predict the next-day exchange rate trend with no need for retraining or other costly computational tasks. We assume that calendar effects would constrain investors’ actions; furthermore, their constrained individual actions would collectively induce deterministic patterns in the financial time-series movement. Hence, financial time-series forecasting models could be prone to monthly repeat their performance patterns, and we could use this information to obtain better predictions and consistently achieve profit. To verify the effectiveness of our methodology, we predicted the sign of the US Dollar to Brazilian Real rate variation. Our proposed models generated a profit metric value 24% higher than the original voting-based ensemble models with 16% lower volatility, gathering two positive elements: higher return with lower risk. The experiments’ outcomes supported the hypothesis that there are considerable improvements with almost no extra computational effort by taking into account behavioral patterns in foreign exchange predictions.