Ensemble Classifier Research Articles

Wavelet decomposition-based multi-stage feature engineering and optimized ensemble classifier for stock market prediction

Expanded feature engineering to handle complex and noisy stock data is not much explored in financial forecasting. Along with feature engineering, hyperparameter tuning is also important. This paper provides a solution to handle these issues by elaborating on Discrete Wavelet Transform (DWT)-based feature engineering and hyperparameter-tuned ensemble model. The Multi-Stage Feature Engineering (MSFE) is proposed in which DWT-based decomposition is utilized to handle the noise. DWT decomposition expands the features; therefore, two-stage feature reduction is proposed in which first the filter method is used, and then the probabilistic method is utilized. Next, hyperparameter tuning of the ensemble model is offered through Particle Swarm Optimization (PSO). The proposed model is named as Wavelet-Particle Swarm Optimization (WPSO). WPSO is tested and evaluated on the three stock indices (NIFTY, NASDAQ, and NYSE), and provided 92.51%, 94.18%, and 87.62% accuracy for NIFTY, NASDAQ, and NYSE, respectively. The WPSO is validated by comparing it with state-of-the-art methods. The performance of the WPSO is statistically analyzed through the Bonferroni–Dunn post hoc test where WPSO positioned at rank 1 for all the evaluation metric and datasets. The WPSO empirically verifies that improving feature quality through MSFE and hyperparameter tuning of ensemble model significantly improves the predictive outcomes.

Fault Identification of UHVDC Transmission Based on DF-AD and Ensemble Learning

High-resistance ground faults are difficult to detect with existing ultrahigh voltage direct current (UHVDC) transmission fault detection systems because of their low sensitivity. To address this challenge, a straightforward mathematical method has been proposed for fault detection in UHVDC system based on the downsampling factor (DF) and approximation derivatives (AD). The signals at multiple sampling frequencies were analysed using the DF, and the AD approach was used to generate various levels of detail and approximation coefficients. Initially, the signals were processed with different DF values. The first, second, and third order derivatives of the generated signals were calculated by the AD method. Next, the entropy features of these signals were computed, and the Random Forest-Recursive feature elimination with cross-validation (RF-RFECV) algorithm was used to select a high-quality feature subset. Finally, an ensemble classifier consisting of Light Gradient Boosting Machine (LightGBM), K Nearest Neighbor (KNN), and Naive Bayes (NB) classifiers was utilized to identify UHVDC faults. The MATLAB/Simulink simulation software was used to develop a ±800 kV UHVDC transmission line model and perform simulation experiments with various fault locations and types. Based on the experiments, it has been established that the suggested approach is highly precise in detecting several faults on UHVDC transmission lines. The method is capable of accurately identifying low or high resistance faults, irrespective of their incidence, and is remarkably resistant to transitional resistance. Furthermore, it exhibits excellent performance in identifying faults using a small sample size and is highly reliable.

Ensemble Classifier Based on Interval Modeling for Microarray Datasets.

The purpose of the study is to propose a multi-class ensemble classifier using interval modeling dedicated to microarray datasets. An approach of creating the uncertainty intervals for the single prediction values of constituent classifiers and then aggregating the obtained intervals with the use of interval-valued aggregation functions is used. The proposed heterogeneous classification employs Random Forest, Support Vector Machines, and Multilayer Perceptron as component classifiers, utilizing cross-entropy to select the optimal classifier. Moreover, orders for intervals are applied to determine the decision class of an object. The applied interval-valued aggregation functions are tested in terms of optimizing the performance of the considered ensemble classifier. The proposed model's quality, superior to other well-known and component classifiers, is validated through comparison, demonstrating the efficacy of cross-entropy in ensemble model construction.

Enhancing Cancer Classification Through Ensemble Machine Learning and Gene Selection Approaches

The vast dimensionality of gene expression data and the limited number of relevant genes necessitate the adoption of gene selection techniques. Furthermore, the choice of an efficient classifier plays a pivotal role in achieving accurate results. In this study, we employ the Minimum Redundancy Maximum Relevance (mRMR) method for gene selection, coupled with ensemble classifiers and individual classifiers like K-Nearest Neighbors (KNN) and Decision Trees (DT). A comparative analysis between two ensemble classifiers and two individual classifiers is conducted, revealing the superior performance of the ensemble classifiers. Our investigation utilizes four distinct cancer gene expression datasets to showcase the efficacy of employing ensemble classifiers and gene selection methods for cancer classification. The ensemble classifier (Bagging Classifier), in conjunction with the MRMR method selecting only the top 30 genes, achieved an impressive overall accuracy of 94% across all four employed datasets.

An ensemble framework-based approach for modeling stability of expansive soil slopes: fusion of machine learning algorithms and protection structure disease data.

Slope failures lead to catastrophic consequences in numerous countries, so accurate slope stability evaluation is critical in geological disaster prevention and control. In this study, the type and characteristics of slope protection structure disease were determined through the field investigation of an expansive soil area, and this information is incorporated into the numerical simulations and works to develop prediction models of slope stability. Four base machine learning (ML) methods are used to capture the relationship between protection structure diseases and factor of safety (FOS). Further, with the help of stacked generalization (SG), four ML models are combined, and the final SG model is used to predict the FOS. The results show that ML methods can effectively utilize this information and achieve excellent prediction results. The proposed SG model exhibits superior accuracy and robustness in predicting FOS compared to other ML methods. With FOS as the regression variable, the main feature contributions are slope height (37.05%) > slip distance of retaining wall (25.43%) > expansive force (18.03%) > slope gradient (12.00%); the coupling relationship among features is also captured by the proposed model. It is concluded that the SG method is particularly suitable for slope stability modeling under small sample conditions. Besides, the SG-based model effectively captures the impact of protection structure diseases on slope stability, enhances the interpretability of the ML model, and provides a reference for the maintenance and repair of the protection structure.

Performance comparison ensemble classifier’s performance in answering frequently asked questions about psychology

In today’s era of digital healthcare transformation, there is a growing demand for swift responses to mental health queries. To meet this need, we introduce an AI-driven chatbot system designed to automatically address frequently asked questions in psychology. Leveraging a range of classifiers including Support Vector Machine (SVM), K-Nearest Neighbors (KNN), and Naïve Bayes, our system extracts insights from expert data sources and employs natural language processing techniques like LDA Topic Modeling and Cosine similarity to generate contextually relevant responses. Through rigorous experimentation, we find that SVM surpasses Naïve Bayes and KNN in accuracy, precision, recall, and F1-score, making it our top choice for constructing the final response system. This research underscores the effectiveness of ensemble classifiers, particularly SVM, in providing accurate and valuable information to enhance mental health support in response to common psychological inquiries.

A noval approach of classification of COVID-19 from chest CT-Scan images using ensemble classifier in combination with cognition based texture features

A noval approach of classification of COVID-19 from chest CT-Scan images using ensemble classifier in combination with cognition based texture features

Online semi-supervised active learning ensemble classification for evolving imbalanced data streams

Concept drift is a core challenge in classification tasks of data streams. Although many drift adaptation methods have been presented, most of them assume that labels of all data are available, which is impractical in many real-world applications. Additionally, the absence of label makes the imbalance ratio of an imbalanced data stream difficultly being obtained in time, providing the inaccurate guidance for resampling and causing poor generalization. To tackle the joint challenges, an online semi-supervised active learning method is proposed to classifier imbalanced data streams with concept drift. A newly-arrived data is first added to the sliding window, and then assigned a pseudo label in terms of its nearest cluster. Meanwhile, semi-supervised clustering algorithm offers its predicted label. Based on the above two predictive labels, cluster-based query strategy provides the criteria for the evaluation and selection of representative instances. More especially, the uncertainty and importance of instances are defined to synthetically evaluate its representativeness. After obtaining true labels of typical ones, ensemble classifier is updated by all instances in current sliding window. Experimental results on 13 synthetic and real data streams indicate that the proposed method outperforms six comparative methods on both G-mean and Recall under various labeling budgets.

An Empirical Study to Analyse The Effect of Bagging and Feature Subspacing on The Performance of A Custom Ensemble Algorithm for Predicting Drug Protein Interactions

Objectives: The objective of this study is to analyse the effect of bagging and feature subspacing on the performance of a custom ensemble of decision tree classifiers for predicting drug protein interactions. Methods: In our present work we have designed a custom ensemble algorithm with decision trees as the base learner. We analysed the effect of bagging negative samples and feature subspacing on the performance of the custom ensemble in terms of AUCROC and AUPR. The Enzyme dataset from the Yamanishi dataset composed of 445 drugs and 664 proteins was used for the experiments. Findings: It was observed that the effect of bagging negative samples was significant as compared to feature supspacing in terms of AUPR metric. Now since AUPR is a metric that remains unaffected by the presence of negative samples hence the increase in AUPR by increasing the negative to positive ratio clearly indicated that the negative samples do contain the positives which are unknown and are yet to be verified. Novelty: The results give a strong indication that that feature subspacing has no considerable impact on the AUCROC metric performance of the custom ensemble while AUPR metric increases as the negative to positive ratio increases. The results give a foundation to the fact that, finding reliable negative samples from the entire set of negative drug protein pairs can further enhance the performance of the machine learning classifiers. Keywords: Decision tree classifier, Ensemble classifier, Drug discovery, Bagging, Drug repurposing

Statistical performance assessment of supervised machine learning algorithms for intrusion detection system

<span lang="EN-US">Several studies have shown that an ensemble classifier's effectiveness is directly correlated with the diversity of its members. However, the algorithms used to build the base learners are one of the issues encountered when using a stacking ensemble. Given the number of options, choosing the best ones might be challenging. In this study, we selected some of the most extensively applied supervised machine learning algorithms and performed a performance evaluation in terms of well-known metrics and validation methods using two internet of things (IoT) intrusion detection datasets, namely network-based anomaly internet of things (N-BaIoT) and internet of things intrusion detection dataset (IoTID20). Friedman and Dunn's tests are used to statistically examine the significant differences between the classifier groups. The goal of this study is to encourage security researchers to develop an intrusion detection system (IDS) using ensemble learning and to propose an appropriate method for selecting diverse base classifiers for a stacking-type ensemble. The performance results indicate that adaptive boosting, and gradient boosting (GB), gradient boosting machines (GBM), light gradient boosting machines (LGBM), extreme gradient boosting (XGB) and deep neural network (DNN) classifiers exhibit better trade-off between the performance parameters and classification time making them ideal choices for developing anomaly-based IDSs.</span>

Interval-based sparse ensemble multi-class classification algorithm for terahertz data

Terahertz time-domain spectroscopy (THz-TDS) has been widely used for food and drug identification. The classification information of a THz spectrum usually does not exist in the whole spectral band but exists only in one or several small intervals. Therefore, feature selection is indispensable in THz-based substance identification. However, most THz-based identification methods empirically intercept the low-frequency band of the THz absorption coefficients for analysis. In order to adaptively find out important intervals of the THz spectra, an interval-based sparse ensemble multi-class classifier (ISEMCC) for THz spectral data classification is proposed. In ISEMCC, the THz spectra are first divided into several small intervals through window sliding. Then the data of training samples in each interval are extracted to train some base classifiers. Finally, a final robust classifier is obtained through a nonnegative sparse combination of these trained base classifiers. With l1 -norm, two objective functions that based on Mean Square Error (MSE) and Cross Entropy (CE) are established. For these two objective functions, two iterative algorithms based on the Alternating Direction Method of Multipliers (ADMM) and Gradient Descent (GD) are built respectively. ISEMCC transforms the problem of interval feature selection and decision-level fusion into a nonnegative sparse optimization problem. The sparse constraint ensures only a few important spectral segments are selected. In order to verify the performance of the proposed algorithm, comparative experiments on identifying the origin of Bupleurum and the harvesting year of Tangerine peel are carried out. The base classifiers used by ISEMCC are Support Vector Machine (SVM) and Decision Tree (DT). The experimental results demonstrate that the proposed algorithm outperforms six typical classifiers, including Random Forest (RF), AdaBoost, RUSBoost, ExtraTree, and the two base classifiers, in terms of classification accuracy.

Adaptive regularized ensemble for evolving data stream classification

Extracting knowledge from data streams requires fast incremental algorithms that are able to handle unlimited processing and ever-changing data with finite memory. A strategy for this challenge is the use of ensembles owing to their ability to tackle concept drift and achieve highly accurate predictions. However, ensembles often require a lot of computational resources. In this study, we propose a novel ensemble-based classification algorithm for data streams, Adaptive Regularized Ensemble (ARE), with low demand for computational resources. The algorithm combines strategies that contribute to high prediction accuracy using only incorrectly classified instances into the training step, random-sized feature subspace for each ensemble element and classifier selection for final ensemble voting. After an extensive experimental study, ARE exhibited high predictive performance and outperformed state-of-the-art ensembles on data streams for real and synthetic datasets while requiring a low processing time and memory consumption.

An ensemble approach for electrocardiogram and lip features based biometric authentication by using grey wolf optimization

In the pursuit of fortified security measures, the convergence of multimodal biometric authentication and ensemble learning techniques have emerged as a pivotal domain of research. This study explores the integration of multimodal biometric authentication and ensemble learning techniques to enhance security. Focusing on lip movement and electrocardiogram (ECG) data, the research combines their distinct characteristics for advanced authentication. Ensemble learning merges diverse models, achieving increased accuracy and resilience in multimodal fusion. Harmonizing lip and ECG modalities establishes a robust authentication system, countering vulnerabilities in unimodal methods. This approach leverages ECG's robustness against spoofing attacks and lip's fine-grained behavioral cues for comprehensive authentication. Ensemble learning techniques, from majority voting to advanced methods, harness the strengths of individual models, improving accuracy, reliability, and generalization. Moreover, ensemble learning detects anomalies, enhancing security. The study incorporates ECG signal filtering and lip region extraction as preprocessing, uses wavelet transform for ECG features, SIFT for lip image features, and employs greywolf optimization for feature selection. Ultimately, a voting-based ensemble classifier is applied for classification, showcasing the potential of this integrated approach in fortified security measures.

Classification of The Human Development Index in Kalimantan Using Random Forest Method

Random Forest is an ensemble classifier in a machine-learning algorithm. The ensemble classifier aimed to improve model accuracy and classification performance. Based on accuracy measures, Random Forest shows the best performance with existing ensemble classifiers like Support Vector Machine (SVM) and AdaBoost. Hence, this research will classify Human Development Index in Kalimantan using a Random Forest classifier. The predictor variables of classification are the average length of schooling, adjusted per capita additions, life expectancy, and length of school expectations. The Random Forest showed that the number of trees selected was 500 and the m being tried was 2. Adjusted per capita additions were the most influential variable in the increase of The Human Development Index with an importance of 10,61%. The Accuracy of classification was 58,33%.

Decoding Autism: Uncovering patterns in brain connectivity through sparsity analysis with rs-fMRI data

Background:In the realm of neuro-disorders, precise diagnosis and treatment rely heavily on objective imaging-based biomarker identification. This study employs a sparsity approach on resting-state fMRI to discern relevant brain region connectivity for predicting Autism. New method:The proposed methodology involves four key steps: (1) Utilizing three probabilistic brain atlases to extract functionally homogeneous brain regions from fMRI data. (2) Employing a hybrid approach of Graphical Lasso and Akaike Information Criteria to optimize sparse inverse covariance matrices for representing the brain functional connectivity. (3) Employing statistical techniques to scrutinize functional brain structures in Autism and Control subjects. (4) Implementing both autoencoder-based feature extraction and entire feature-based approach coupled with AI-based learning classifiers to predict Autism. Results:The ensemble classifier with the extracted feature set achieves a classification accuracy of 84.7% ± 0.3% using the MSDL atlas. Meanwhile, the 1D-CNN model, employing all features, exhibits superior classification accuracy of 88.6% ± 1.7% with the Smith 2009 (rsn70) atlas. Comparison with existing method (s):The proposed methodology outperforms the conventional correlation-based functional connectivity approach with a notably high prediction accuracy of more than 88%, whereas considering all direct and noisy indirect region-based functional connectivity, the traditional methods bound the prediction accuracy within 70% to 79%. Conclusions:This study underscores the potential of sparsity-based FC analysis using rs-fMRI data as a prognostic biomarker for detecting Autism.

ENSEMBLE CLASSIFIER BASED FAKE NEWS IDENTIFICATION IN ONLINE SOCIAL NETWORK

In today’s context, one of the main mediums of news consumption is through social media. It has become a common trend now to produce fake news for speedy propagation and popularity. Thus, it creates a sort of illusion and deception for readers. Amidst the umpty number of researches done, we could say that there is no research that could accurately predict fake news over online. In this study, the ensemble classifier is employed to develop a model for fake news identification in online social networks. This proposed method applies the stacked ensemble classification model, the proposed approach that learns the represented text model and classifies the textual data into real news and fake news. Then, the proposed approach takes the collaborative decision from the classified data that is generated from the multiple base learners using the weight-based ensemble method. The accuracy prediction and performance evaluation of time consumption of detection fake news are consecutively 80 Percentage and 11 ms respectively. Thus, it improves the efficiency and classification accuracy over large-scale social media messages through an efficient sentiment analysis model.

Unmasking Deception: A Proactive Approach to Detecting and Preventing Fake Job Offers

The research presents an innovative approach to combating fraudulent job postings on the internet through the utilization of machine learning-based classification techniques. By leveraging different classifiers, including single classifiers and ensemble classifiers, the study aims to discern fake job postings amidst a vast array of online listings. Through an extensive analysis of experimental results, the research identifies ensemble classifiers as the optimal choice for detecting employment scams, surpassing the efficacy of single classifiers. Employing a dataset sourced from Kaggle, the study focuses on distinguishing between real and fake job postings, with the latter comprising a minority of the dataset, as anticipated. By adhering to these structured stages, the research aims to contribute to the advancement of methods for identifying and mitigating fraudulent activities in online job postings, thereby enhancing the integrity of online recruitment processes.

Enhancing software defect prediction: a framework with improved feature selection and ensemble machine learning

Effective software defect prediction is a crucial aspect of software quality assurance, enabling the identification of defective modules before the testing phase. This study aims to propose a comprehensive five-stage framework for software defect prediction, addressing the current challenges in the field. The first stage involves selecting a cleaned version of NASA’s defect datasets, including CM1, JM1, MC2, MW1, PC1, PC3, and PC4, ensuring the data’s integrity. In the second stage, a feature selection technique based on the genetic algorithm is applied to identify the optimal subset of features. In the third stage, three heterogeneous binary classifiers, namely random forest, support vector machine, and naïve Bayes, are implemented as base classifiers. Through iterative tuning, the classifiers are optimized to achieve the highest level of accuracy individually. In the fourth stage, an ensemble machine-learning technique known as voting is applied as a master classifier, leveraging the collective decision-making power of the base classifiers. The final stage evaluates the performance of the proposed framework using five widely recognized performance evaluation measures: precision, recall, accuracy, F-measure, and area under the curve. Experimental results demonstrate that the proposed framework outperforms state-of-the-art ensemble and base classifiers employed in software defect prediction and achieves a maximum accuracy of 95.1%, showing its effectiveness in accurately identifying software defects. The framework also evaluates its efficiency by calculating execution times. Notably, it exhibits enhanced efficiency, significantly reducing the execution times during the training and testing phases by an average of 51.52% and 52.31%, respectively. This reduction contributes to a more computationally economical solution for accurate software defect prediction.

Improve quality of service for the Internet of Things using Blockchain & machine learning algorithms

The quality of service (QoS) parameters in IoT applications plays a prominent role in determining the performance of an application. Considering the significance and popularity of IoT systems, it can be predicted that the number of users and IoT devices are going to increase exponentially shortly. Therefore, it is extremely important to improve the QoS provided by IoT applications to increase their adaptability. Majority of the IoT systems are characterized by their heterogeneous and diverse nature. It is challenging for these systems to provide high-quality access to all the connecting devices with uninterrupted connectivity. Considering their heterogeneity, it is equally difficult to achieve better QoS parameters. Artificial intelligence-based machine learning (ML) tools are considered a potential tool for improving the QoS parameters in IoT applications. This research proposes a novel approach for enhancing QoS parameters in IoT using ML and Blockchain techniques. The IoT network with Blockchain technology is simulated using an NS2 simulator. Different QoS parameters such as delay, throughput, packet delivery ratio, and packet drop are analyzed. The obtained QoS values are classified using different ML models such as Naive Bayes (NB), Decision Tree (DT), and Ensemble, learning techniques. Results show that the Ensemble classifier achieves the highest classification accuracy of 83.74 % compared to NB and DT classifiers.

Accurate, Uncertainty-Aware Classification of Molecular Chemical Motifs from Multimodal X-ray Absorption Spectroscopy.

Accurate classification of molecular chemical motifs from experimental measurement is an important problem in molecular physics, chemistry, and biology. In this work, we present neural network ensemble classifiers for predicting the presence (or lack thereof) of 41 different chemical motifs on small molecules from simulated C, N, and O K-edge X-ray absorption near-edge structure (XANES) spectra. Our classifiers not only achieve class-balanced accuracies of more than 0.95 but also accurately quantify uncertainty. We also show that including multiple XANES modalities improves predictions notably on average, demonstrating a "multimodal advantage" over any single modality. In addition to structure refinement, our approach can be generalized to broad applications with molecular design pipelines.