Base Classifiers Research Articles

Fusing multiple interval-valued fuzzy monotonic decision trees

As a powerful knowledge mining technique for ordinal classification tasks, dominance-based rough set theory has many advantages but also some issues. Sensitivity to noisy information means that even a single mislabeled sample can lead to substantial fluctuations in the approximation calculations. Another issue is that most monotonic classifiers can only handle real-valued data and cannot directly deal with interval-valued data, which is common in practical applications. To address these issues, a tree-based fusion learning method for monotonic classification of interval-valued attributes, named FM-IFMDT, has been proposed. Its functional structure comprises three components: (i) The proposed robust β-precision interval-valued fuzzy dominance neighborhood rough set model (β-IFDNRS) can adaptively identify and filter noise samples. (ii) The interval dominance discernibility matrix based on β-IFDNRS is developed for feature selection and can generate a set of complete and diverse feature subsets. (iii) The novel interval-valued fuzzy monotonic decision tree (IFMDT) based on the probability distribution is trained on each feature subset and is used as the base classifier of the weighted voting fusion model. Extensive experiments show the proposed fusion learning method has significant advantages.

МЕТОД ІНТЕГРУВАННЯ ДОМЕННИХ ЗНАНЬ У БАГАТОСТРАТЕГІЧНУ КЛАСИФІКАЦІЮ МЕДИЧНИХ ЗОБРАЖЕНЬ

The integration of domain-specific knowledge into medical image classification systems presents a promising approach to enhance the accuracy and interpretability of diagnostic models, particularly in the field of medical imaging. This study aimed to evaluate the efficacy of incorporating domain knowledge into multi-strategic classification algorithms for medical images, with a focus on magnetic resonance imaging (MRI) of the brain. This was a comparative, explorative study utilizing a newly developed multi-strategic classification framework that integrates medical domain knowledge into the classification process. The dataset comprised MRI scans of the brain, collected from three different medical institutions, totaling 320 images, which were annotated by expert radiologists. The framework employed basic classifiers, where each classifier was trained on examples incorrectly classified by another, fostering a collaborative learning environment. Performance was compared against traditional single-strategy classification methods. The integrated multi-strategic classification model demonstrated superior performance, achieving higher sensitivity, specificity, and accuracy with 98 %, 97 %, and 96 %, respectively. Furthermore, the model significantly improved the interpretability of diagnostic predictions, aligning closely with expert radiologists' assessments. The incorporation of domain-specific knowledge into the classification of medical images significantly enhances both the accuracy and the interpretability of the results. The multi-strategic approach allows for a more robust and nuanced understanding of complex medical images, such as MRI scans of the brain, potentially leading to improved diagnostic processes and outcomes in clinical settings. Future research should focus on expanding the types of medical images analyzed and further refining the integration techniques to encompass a broader spectrum of domain knowledge.

Tree-based Machine Learning Ensembles and Feature Importance Approach for the Identification of Intrusions in UNR-IDD Dataset

Detection of intrusions from network data with the use of machine learning techniques has gained great attention in the past decades. One of the key problems in the network security domain is the availability of representative datasets for testing and evaluation purposes. Despite several efforts by researchers to release datasets that can be used for benchmarking attack detection models, some of the released datasets still suffer from one limitation or the other. Thus, some researchers at the University of Nevada released a dataset named UNR-IDD dataset which was argued to be free from some of the limitations of the past datasets. This study proposed Tree-based ensemble approaches for building binary intrusion identification models from the UNR-IDD dataset. Decision Tree algorithms are used as base classifiers in the Extra Trees, Random Forest and AdaBoost-based intrusion detection models. The results of the experimental analyses carried out indicated that the three ensembles performed excellently when feature selection was used compared to when all features were applied. For instance, Extra Trees model achieved an accuracy of 0.97, precision of 0.98, recall of 0.98 and f1-score of 0.98. Similarly, Random Forest model achieved an accuracy of 0.98, precision of 0.98, recall of 0.99 and f1-score of 0.98. Adaboost-based model had an accuracy of 0.96, precision of 0.96, recall of 0.99 and f1-score of 0.98. It was deduced that Random Forest intrusion classification model achieved slight overall best results when compared to the other models built. It is concluded that the three homogeneous ensemble models achieved very promising results while feature importance was used as attribute selection method.

Feature engineering to identify plant diseases using image processing and artificial intelligence: A comprehensive review

Plant diseases can significantly reduce crop yield and product quality. Visual inspections of plants by human observers for disease identification are time-consuming, costly, and prone to error. Advances in artificial intelligence (AI) have created opportunities for the rapid diagnosis and non-destructive classification of plant pathogens. Several machine vision techniques have been developed to identify and classify plant diseases automatically based on the morphology of specific symptoms. The use of deep learning models has achieved acceptable disease classification results, but they require large datasets for training, which can be labor-intensive, time-consuming, and computationally costly This problem can be solved, to a point, by using data augmentation techniques and generative AI in order to increase the size of the datasets. Furthermore, a combination of deep feature extraction and classification by machine learning was used for accurate disease detection and classification. In some cases, traditional base classifiers trained with small datasets including basic shape, color, and texture features can be feasible for the efficient identification of plant diseases. The performance of such classifiers depends primarily on the features extracted from images; therefore, feature extraction plays a vital role in identifying diseases. Feature engineering, a process to identify the most relevant variables from raw data in order to develop an efficient predictive model, is explored in this paper.

Machine learning techniques in classifying satisfaction with the economy of Latin American Citizens

Applications of machine learning algorithms offer a new perspective for exploring data from opinion polls, aiming to better understand perceptions and population profiles on various topics, including satisfaction with the economy. In this context, this research aimed to apply machine learning algorithms to classify economic satisfaction from citizens in Latin American countries, evaluating their performance on different datasets and analyzing the variables contributing most to this theme. Four base algorithms were applied: Random Forest; Gradient Boosting; XGBoost; and Naïve Bayes, along with two ensemble methods: hard voting; and soft voting. Subsequently, variable importance was assessed using Gradient Boosting and Random Forest methods, showing the contribution of the predictors to the economic satisfaction. The results showed an accuracy of approximately 86% for three of the base classifiers and soft voting but all methods performed better in classifying dissatisfied citizens, struggling to recognize patterns of those satisfied with the economy. The most important predictor variables considered for classifying this satisfaction were "Satisfaction with democracy" and "Comparison with the economy 12 months before".

Classification of bioactive peptides: A systematic benchmark of models and encodings

Bioactive peptides are short amino acid chains possessing biological activity and exerting physiological effects relevant to human health. Despite their therapeutic value, their identification remains a major problem, as it mainly relies on time-consuming in vitro tests. While bioinformatic tools for the identification of bioactive peptides are available, they are focused on specific functional classes and have not been systematically tested on realistic settings. To tackle this problem, bioactive peptide sequences and functions were here gathered from a variety of databases to generate a unified collection of bioactive peptides from microbial fermentation. This collection was organized into nine functional classes including some previously studied and some unexplored such as immunomodulatory, opioid and cardiovascular peptides. Upon assessing their sequence properties, four alternative encoding methods were tested in combination with a multitude of machine learning algorithms, from basic classifiers like logistic regression to advanced algorithms like BERT. Tests on a total of 171 models showed that, while some functions are intrinsically easier to detect, no single combination of classifiers and encoders worked universally well for all classes. For this reason, we unified all the best individual models for each class and generated CICERON (Classification of bIoaCtive pEptides fRom micrObial fermeNtation), a classification tool for the functional classification of peptides. State-of-the-art classifiers were found to underperform on our realistic benchmark dataset compared to the models included in CICERON. Altogether, our work provides a tool for real-world peptide classification and can serve as a benchmark for future model development.

Adap-BDCM: Adaptive Bilinear Dynamic Cascade Model for Classification Tasks on CNV Datasets.

Copy number variation (CNV) is an essential genetic driving factor of cancer formation and progression, making intelligent classification based on CNV feasible. However, there are a few challenges in the current machine learning and deep learning methods, such as the design of base classifier combination schemes in ensemble methods and the selection of layers of neural networks, which often result in low accuracy. Therefore, an adaptive bilinear dynamic cascade model (Adap-BDCM) is developed to further enhance the accuracy and applicability of these methods for intelligent classification on CNV datasets. In this model, a feature selection module is introduced to mitigate the interference of redundant information, and a bilinear model based on the gated attention mechanism is proposed to extract more beneficial deep fusion features. Furthermore, an adaptive base classifier selection scheme is designed to overcome the difficulty of manually designing base classifier combinations and enhance the applicability of the model. Lastly, a novel feature fusion scheme with an attribute recall submoduleis constructed, effectively avoiding getting stuck in local solutions and missing some valuable information. Numerous experiments have demonstrated that our Adap-BDCM model exhibits optimal performance in cancer classification, stage prediction, and recurrence on CNV datasets. This study can assist physicians in making diagnoses faster and better.

Autism spectrum disorder identification using multi‐model deep ensemble classifier with transfer learning

AbstractIdentifying autism spectrum disorder (ASD) symptoms accurately is a challenging task. The traditional subjective diagnostic process of ASD relies on time‐consuming behavioural and psychological observations. In this study, we introduce an ensemble learning‐based classification model using an open‐access database focusing on functional magnetic resonance imaging (fMRI). We propose a novel multi‐model ensemble classifier (MMEC) and multisite ensemble classifier (MSEC) with transfer learning (TL) for ASD classification to improve the prediction accuracy. The MMEC utilizes four base classifiers, Inception V3, ResNet50, MobileNet, and DenseNet to boost the performance of the individual convolutional neural network (CNN) models. The MSEC combined the base classifiers trained from different data sites. We evaluate the two models with ensemble averaging, weighted averaging, and stacking methods. The proposed MMEC with stacking shows the state of art performance compared to MSEC, improving the prediction accuracy by 3.25%. The obtained results have shown an accuracy of 97.82%, 97.82%, and 97.78% for ensemble averaging, weighted averaging, and stacking methods, respectively, on multi‐site datasets. The ensemble classifier MMEC performed better than a single classifier on the multi‐site dataset. The proposed MMEC opens a new paradigm to design a universal ASD classification framework.

Enhancing educational evaluation through predictive student assessment modeling

This study evaluates several machine learning models used in predicting student performance. The data utilized in this study was collected from 253 undergraduate students participating in five classes within one of three courses offered by VnCodelab, an interactive learning management system, to provide insights into student performance. Leveraging the data-rich environment of the interactive learning management system proposed earlier, this study focuses on training a predictive model that forecasts student grades based on the comprehensive data collected during the teaching process. The proposed model capitalizes on the data obtained from students' engagement patterns, time spent on exercises, and progress tracking across learning activities. This study compared five different base classifiers— Random Forest (RF), Logistic Regression (LR), Support Vector Machine (SVM), Naïve Bayes (NB), and k-nearest Neighbor (k-NN), and an ensemble learning method Stacking Classifier —utilizing a dataset comprising 13 features. The research assesses the model's accuracy, reliability, and implications, contributing to the evolution of educational evaluation by introducing predictive assessment as a transformative tool. The results indicate that the Stacking Classifier accurately predicts students' grade ranges, surpassing individual base classification models by effectively combining their predictive capabilities. Integrating data-driven forecasting into the educational ecosystem can transform teaching methodologies and foster an informed, engaged, and empowered learning environment. This approach cultivates a proactive learning community by empowering students with real-time academic progress forecasts. Educators benefit from data-informed insights that facilitate more effective and objective performance evaluation.

A fault diagnosis framework based on heterogeneous ensemble learning for air conditioning chiller with unbalanced samples

Big data-based air conditioning fault diagnosis research has developed rapidly in recent years, but in actual engineering, the fault sample size of air conditioning systems is much smaller than the normal sample size, and the resulting sample imbalance problem makes conventional data-driven diagnostic methods based on low accuracy and poor stability. In order to solve the problem of unbalanced fault diagnosis of air-conditioning chillers, this paper proposes an integrated learning-based diagnostic model, which achieves diagnosis by combining multiple base models and by majority voting. The method uses four classification models, namely, random forest model, decision tree model, k nearest neighbor model, and isomorphic integration model, as base classifiers, and synthesizes the four base classifiers into a heterogeneous integration algorithmic model (IMV) through integrated learning, and performs diagnostic detection of seven types of typical faults of chiller units using the majority voting method of integrated learning. The effectiveness of the proposed algorithm is verified on the RP-1043 dataset, and the experimental results show that the accuracy of the heterogeneous integrated algorithm model (IMV) can reach 96.87%, which is a significant improvement compared with the accuracy of the other four base classifier models (81.04%–96.25%). Therefore, the integrated learning model has some application prospects in fault diagnosis when targeting unbalanced datasets.

Cancer Classification from Gene Expression Using Ensemble Learning with an Influential Feature Selection Technique

Uncontrolled abnormal cell growth, known as cancer, may lead to tumors, immune system deterioration, and other fatal disability. Early cancer identification makes cancer treatment easier and increases the recovery rate, resulting in less mortality. Gene expression data play a crucial role in cancer classification at an early stage. Accurate cancer classification is a complex and challenging task due to the high-dimensional nature of the gene expression data relative to the small sample size. This research proposes using a dimensionality-reduction technique to address this limitation. Specifically, the mutual information (MI) technique is first utilized to select influential biomarker genes. Next, an ensemble learning model is applied to the reduced dataset using only the most influential features (genes) to develop an effective cancer classification model. The bagging method, where the base classifiers are Multilayer Perceptrons (MLPs), is chosen as an ensemble technique. The proposed cancer classification model, the MI-Bagging method, is applied to several benchmark gene expression datasets containing distinctive cancer classes. The cancer classification accuracy of the proposed model is compared with the relevant existing methods. The experimental results indicate that the proposed model outperforms the existing methods, and it is effective and competent for cancer classification despite the limited size of gene expression data with high dimensionality. The highest accuracy achieved by the proposed method demonstrates that the proposed emerging gene-expression-based cancer classifier has the potential to help in cancer treatment and lead to a higher cancer survival rate in the future.

Graph ensemble neural network

Ensemble methods have been shown to improve graph neural networks (GNNs). Existing ensemble methods on graphs determine a strong classifier by combining a set of trained base classifiers, i.e., combining the final outputs of base classifiers for prediction. However, these methods fail to promote many popular GNNs to perform well under heterophily (in graphs where many connected nodes have different class labels), which limits their applicability. Furthermore, they ignore the hierarchical nature of GNNs, which results in no interaction between base classifiers when neighbors are aggregated (during training). Two issues arise from this: low applicability and shallow ensemble. We propose Graph Ensemble Neural Network (GEN) for addressing above issues, which is not a simple ensemble of GNNs, but instead integrates ensemble into GNNs to fuse a set of graphs. GEN deepens single ensemble into multiple ensembles during training and applies to homophily and heterophily graphs. In GEN, we design structure augmentation to generate some graphs for training and design feature augmentation for attenuating errors brought by the initial features. Different from existing graph ensemble methods that execute only one ensemble, GEN executes multiple deep ensembles throughout the neighbor aggregation to fuse multiple graphs generated by structure augmentation. Extensive experiments show that GEN achieves new state-of-the-art performance on homophily and heterophily graphs for the semi- and full-supervised node classification. The source code of GEN is publicly available at https://github.com/graphNN/GEN1.

Tuberculosis Detection and Air Purifier Suggestion

Tuberculosis (TB) is a disease that can be fatal if not promptly treated. Ensemble deep learning methods have shown promise in aiding the early detection of TB. Previous research typically trained ensemble classifiers using images with similar features, but for optimal performance, an ensemble requires a range of errors, achievable through diverse classification techniques or feature sets. This study focuses on the latter approach, presenting TB detection using deep learning alongside contrast-enhanced canny edge detected (CEED-Canny) X-ray images. CEED-Canny was employed to generate edge-detected lung X-ray images. Two sets of features were derived: one from the enhanced X- ray images and the other from the edge-detected images. By introducing this variation in features, the diversity of errors among the base classifiers was increased, resulting in improved TB detection. The proposed ensemble method achieved a comparable accuracy of 93.59%, sensitivity of 92.31%, and specificity of 94.87% compared to prior research.

Online Muscle Activation Onset Detection Using Likelihood of Conditional Heteroskedasticity of Electromyography Signals.

The real-time, accurate detection of muscle activation onset (MAO) is significant for EMG-triggered control strategies in embedded applications like prostheses and exoskeletons. This paper investigates sEMG signals using the generalized autoregressive conditional heteroskedasticity (GARCH) model, focusing on variance. A novel feature, the likelihood of conditional heteroskedasticity (LCH) extracted from the maximum likelihood estimation of GARCH parameters, is proposed. This feature effectively distinguishes signal from noise based on heteroskedasticity, allowing for the detection of MAO through the LCH feature and a basic threshold classifier. For online calculation, the model parameter estimation is simplified, enabling direct calculation of the LCH value using fixed parameters. The proposed method was validated on two open-source datasets and demonstrated superior performance over existing methods. The mean absolute error of onset detection, compared with visual detection results, is approximately 65 ms under online conditions, showcasing high accuracy, universality, and noise insensitivity. The results indicate that the proposed method using the LCH feature from the GARCH model is highly effective for real-time detection of muscle activation onset in sEMG signals. This novel approach shows great potential and possibility for real-world applications, reflecting its superior performance in accuracy, universality, and insensitivity to noise.

Optimization Method Based on Machine Learning for College Students’ Psychological Control Source Propensity Classification

Abstract College students tend to have more locus of control, which is greatly affected by college students, resulting in higher classification error rate and longer classification time. An optimized method for classifying the tendency of college students’ locus of control tendency based on machine learning is proposed in this article. Collect the data of college students’ locus of control tendency, build an emotional dictionary based on it, and extract the emotional words and text features from it. According to the feature extraction results, the support vector machine is used to build a base classifier to obtain the preliminary classification results. The deep belief network is used to optimize the preliminary classification results of college students’ locus of control tendency, and the final optimization results of college students’ locus of control tendency classification are obtained. The experimental results show that the error rate of college students’ locus of control tendency classification is −1∼1 %, the average recall rate is 96.2 %, and the average classification time is 0.7 s.

SStackGNN: Graph Data Augmentation Simplified Stacking Graph Neural Network for Twitter Bot Detection

Ensemble learning has the potential to enhance the efficacy of feeble classifiers significantly and is increasingly being utilized in Twitter bot detection. Previous methods have utilized stacking techniques to train the primary classifiers, implementing cross-validation to mitigate overfitting and enhance predictive accuracy. However, cross-validation substantially amplifies the computation time associated with stacking. To overcome this challenge, this paper presents a novel approach, the Simplified Stacking Graph Neural Network (SStackGNN), specifically designed for the detection of social bots. Our methodology leverages the power of Graph Neural Networks (GNNs) as base classifiers, enabling effective capturing of inter-account interactions. In addition, a Multilayer Perceptron (MLP) serves as a secondary classifier, amalgamating the outcomes of the foundational classifiers to generate final predictions, thereby enhancing the predictive performance of the GNNs. Instead of relying on cross-validation and distinct base classifier structures, we employ node-level, edge-level, and feature-level graph data augmentation techniques to acquire diverse foundational classifiers. This approach significantly alleviates the computational complexity while achieving superior performance. Experimental results demonstrate that our proposed SStackGNN outperforms other approaches.

Ensemble learning prediction framework for EGFR amplification status of glioma based on terahertz spectral features

Epidermal growth factor receptor (EGFR) plays a pivotal role in the initiation and progression of gliomas. In particular, in glioblastoma, EGFR amplification emerges as a catalyst for invasion, proliferation, and resistance to radiotherapy and chemotherapy. Current approaches are not capable of providing rapid diagnostic results of molecular pathology. In this study, we propose a terahertz spectroscopic approach for predicting the EGFR amplification status of gliomas for the first time. A machine learning model was constructed using the terahertz response of the measured glioma tissues, including the absorption coefficient, refractive index, and dielectric loss tangent. The novelty of our model is the integration of three classical base classifiers, i.e., support vector machine, random forest, and extreme gradient boosting. The ensemble learning method combines the advantages of various base classifiers, this model has more generalization ability. The effectiveness of the proposed method was validated by applying an individual test set. The optimal performance of the integrated algorithm was verified with an area under the curve (AUC) maximum of 85.8 %. This signifies a significant stride toward more effective and rapid diagnostic tools for guiding postoperative therapy in gliomas.

Deep learning based crop-type mapping using SAR and optical data fusion

Accurate crop maps are essential in various applications related to food security management activities. Remote Sensing is the primary data source for land-use and land-cover monitoring applications. However, high spectral, spatial, and temporal variations of crop types during their phonological stages make crop mapping challenging. According to the complementary behaviors of optical and radar data, a deep learning-based feature-level fusion strategy is proposed for accurate crop-type mapping. The proposed method uses Sparse Auto-Encoders (SAE) as its base classifier and can involve spatial information in two data-preparation and post-processing steps. Involving spatial information in the classification process can significantly improve crop mapping. The post-processing step uses the guided filter as an edge-aware filtering process that uses field boundary information that implicitly exists in data. Accuracy analysis was performed using two pairs of RapidEye and UAVSAR data from two agricultural areas in Canada. The suitability and higher performance of the proposed method were demonstrated compared to traditional machine learning and common decision-level fusion methods. Accuracy metrics were improved by at least 3% and 1% in two datasets. Class F-scores were also significantly enhanced in oat and wheat classes (up to 20%).

Enhancing Academic Success Prediction: An Ensemble Approach

The process of extracting meaning and knowledge from large volumes of data is termed as Data mining. It also refers to the way of inferring information from databases, which can be used in diverse fields including educational domain. Educational data mining plays a key role in finding ways to discover knowledge from data in the education sector. Educational Data Mining has evolved as a significant element in prediction of students’ academic performance. The most important goal of the paper is to analyze and evaluate the engineering students’ performance by applying stacking, an ensemble method in orange tool. Ensemble Stacking combines several base classifier models in order to create one optimal prediction model. Engineering students’ dataset was used to build predictive model using traditional classifiers SVM, Logistic Regression, Naïve Bayes and then stacking technique was implemented. The results showed that the proposed stacking technique obtains a high performance, which has a superior result compared to the other base classifiers techniques. Therefore, conclusion could be reached that the stacking performance is better than that of different algorithms.

New ensemble learning algorithm based on classification certainty and semantic correlation

A new bootstrap-aggregating (bagging) ensemble learning algorithm is proposed based on classification certainty and semantic correlation to improve the classification accuracy of ensemble learning. First, two predetermined thresholds are introduced to construct the long and short-text sample subsets, and different deep learning methods are compared to construct the optimal base classifier groups for each sample subsets. Then, the random sampling method employed in traditional bagging classification algorithms is improved, and a threshold group based random sampling method is proposed to obtain long and short training sample subsets of each iteration. Finally, the sample classification certainty of the base classifiers for different categories is defined, and the semantic correlation information is integrated with the traditional weighted voting classifier ensemble method to avoid the loss of important information during the sampling process. The experimental results on multiple datasets demonstrate that the algorithm significantly improves text classification accuracy and outperforms typical deep learning algorithms. The proposed algorithm achieves the improvements of approximately 0.082, 0.061 and 0.019 on CNews dataset when the F1 measurement is used over the traditional ensemble learning algorithms such as random forest, M_ADA_A_SMV and CNN_SVM_LR. Moreover, it achieves the best F1 values of 0.995, 0.985, and 0.989 on the datasets of Spam, CNews, and SogouCS datasets, respectively, when compared with the ensemble learning algorithms using different base classifiers.