Ensemble Classification Framework Research Articles

Hybrid optimization assisted deep ensemble classification framework for skin cancer detection

Hybrid optimization assisted deep ensemble classification framework for skin cancer detection

Explainable feature selection and ensemble classification via feature polarity

Feature selection aims to choose the most relevant features from the dataset that can enhance the performance and efficiency of machine learning models. Although feature selection has been studied for many years, most existing methods focus on accuracy and efficiency while neglecting the interpretability of selected features. Therefore, inspired by the “Yin-Yang” philosophy, we introduce the concept of feature polarity for the first time and divide the features into positive and negative features. For example, by analyzing a patient's symptoms (features), we can obtain two sets of features to explain whether the patient has the flu. Positive features help us determine if the patient has the flu, while negative features can help us rule out the possibility of the flu. We introduce the PN (Positive and Negative) coefficient to measure the polarity of candidate features and develop a novel and explainable feature selection method based on feature polarity. Furthermore, we propose an ensemble classification framework that leverages both positive and negative features for each class to improve classification performance. Extensive experiments demonstrate the effectiveness of the PN coefficient compared to other information measurements. Moreover, our proposed classification framework performs excellently compared to some state-of-the-art feature selection methods.

An effective ensemble classification framework for intrusion detection system using LSTM autoencoder

In intrusion detection, the curse of dimensionality and the trade-off between maintaining a low false alarm rate and achieving a high detection rate are significant challenges. This research suggests a unique strategy based on dimensionality reduction methods to improve the performance of network intrusion detection systems (NIDS). Compressing high-dimensional network traffic data using a Long Short-Term Memory Autoencoder (LSTMAE) allows the reduced characteristics to be submitted to a classifier to identify anomalies that may indicate an attack. Using standard datasets, including Network Security Laboratory - Knowledge Discovery in Datasets (NSL-KDD), UNSW-NB15, and Canadian Institute for Cyber Security - Intrusion Detection Systems (CICIDS2017), the proposed model is tested with classifiers like Random Forest (RF) and LightGBM (Light Gradient Boosting Machines). It is hoped that by adopting this method, NIDS response times may be improved while costs associated with storing and processing data are minimized. Precision, recall, F-score, accuracy, detection rate (DR), and false alarm rate (FAR) are only a few of the performance measures used to assess the quality of the suggested models. The experimental findings show that the proposed LSTMAE model reduces prediction errors more effectively than classic machine learning techniques such as Random Forest (RF), Gradient Boosting (GB), Support Vector Machines (SVM), Deep Belief Networks (DBN), Deep Neural Networks (DNN), Autoencoder (AE), and Long Short-Term Memory (LSTM). The results also show that the proposed solution outperforms the state-of-the-art methods of detection accuracy and computing complexity using accuracy, precision, recall, F1_Score, detection rate, and FAR.

Localization of Dielectric Anomalies with Multi-level Outlier Detection through Membership Function and Ensemble Classification Framework

This research presents an innovative method for real-time detection of dielectric anomalies, with a primary focus on evaluating apple quality and ripeness using dielectric tomography. The study involves the development of an advanced tomography system within an anechoic chamber, harnessing electromagnetic wave technology and sophisticated antenna systems for data acquisition. The proposed framework encompasses critical stages, including data collection, range bounds computation, threshold determination, class membership assignment, and ensemble classification. By seamlessly integrating statistical methods, density-based clustering, and ensemble learning, this approach significantly enhances precision and reliability in anomaly detection. The integration of available statistical methods, density-based clustering, and ensemble learning may demand substantial computational resources, limiting the scalability and real-time applicability of the proposed framework. Empirical results demonstrate the superior performance of the method, with an accuracy rate of 98.9%, precision of 0.989, F-measure of 0.989, dielectric anomaly recall rate of 0.99, and a low error rate of 0.18. Overall, this research introduces an advanced approach with the potential to revolutionize apple quality assessment and industrial processes across various sectors.

Towards unbalanced multiclass intrusion detection with hybrid sampling methods and ensemble classification

Intrusion Detection Systems (IDS) play a crucial role in securing computer networks against malicious activities. However, their efficacy is consistently hindered by the persistent challenge of class imbalance in real-world datasets. While various methods, such as resampling techniques, ensemble methods, cost-sensitive learning, data augmentation, and so on, have individually addressed imbalance classification issues, there exists a notable gap in the literature for effective hybrid methodologies aimed at enhancing IDS performance. To bridge this gap, our research introduces an innovative methodology that integrates hybrid undersampling and oversampling strategies within an ensemble classification framework. This novel approach is designed to harmonize dataset distributions and optimize IDS performance, particularly in intricate multi-class scenarios. In-depth evaluations were conducted using well-established intrusion detection datasets, including the Car Hacking: Attack and Defense Challenge 2020 (CHADC2020) and IoTID20. Our results showcase the remarkable efficacy of the proposed methodology, revealing significant improvements in precision, recall, and F1-score metrics. Notably, the hybrid-ensemble method demonstrated an exemplary average F1 score exceeding 98% for both datasets, underscoring its exceptional capability to substantially enhance intrusion detection accuracy. In summary, this research represents a significant contribution to the field of IDS, providing a robust solution to the pervasive challenge of class imbalance. The hybrid framework not only strengthens IDS efficacy but also illuminates the seamless integration of undersampling and oversampling within ensemble classifiers, paving the way for fortified network defenses.

Assessing the complementary information from an increased number of biologically relevant features in liquid biopsy-derived RNA-Seq data

Liquid biopsy-derived RNA sequencing (lbRNA-seq) exhibits significant promise for clinic-oriented cancer diagnostics due to its non-invasiveness and ease of repeatability. Despite substantial advancements, obstacles like technical artefacts and process standardisation impede seamless clinical integration. Alongside addressing technical aspects such as normalising fluctuating low-input material and establishing a standardised clinical workflow, the lack of result validation using independent datasets remains a critical factor contributing to the often low reproducibility of liquid biopsy-detected biomarkers.Considering the outlined drawbacks, our objective was to establish a workflow/methodology characterised by: 1. Harness the rich diversity of biological features accessible through lbRNA-seq data, encompassing a holistic range of molecular and functional attributes. These components are seamlessly integrated via a Machine Learning-based Ensemble Classification framework, enabling a unified and comprehensive analysis of the intricate information encoded within the data. 2. Implementing and rigorously benchmarking intra-sample normalisation methods to heighten their relevance within clinical settings. 3. Thoroughly assessing its efficacy across independent test sets to ascertain its robustness and potential utility.Using ten datasets from several studies comprising three different sources of biological material, we first show that while the best-performing normalisation methods depend strongly on the dataset and coupled Machine Learning method, the rather simple Counts Per Million method is generally very robust, showing comparable performance to cross-sample methods. Subsequently, we demonstrate that the innovative biofeature types introduced in this study, such as the Fraction of Canonical Transcript, harbour complementary information. Consequently, their inclusion consistently enhances prediction power compared to models relying solely on gene expression-based biofeatures. Finally, we demonstrate that the workflow is robust on completely independent datasets, generally from different labs and/or different protocols. Taken together, the workflow presented here outperforms generally employed methods in prediction accuracy and may hold potential for clinical diagnostics application due to its specific design.

A Metadata-Assisted Cascading Ensemble Classification Framework for Automatic Annotation of Open IoT Data

Public Internet of Things (IoT) platforms, such as Thingspeak, significantly increased the availability of open IoT data and enabled faster and cheaper development of novel IoT applications by reducing or even eliminating the need for deploying their own IoT sensors and platforms. However, open IoT data is often heterogeneous, sparse, fuzzy, and lacks accurate description (which we refer to as IoT metadata). These limitations make open IoT data challenging to integrate and use, and prevent the efficient development of IoT applications. In fact, while several sensor data description models have been proposed and standardized, open IoT data currently lack or include only partial metadata description. Therefore, novel techniques for automatically annotating open IoT data are needed to fully unleash the power of open IoT. This paper proposes a novel Metadata-Assisted Cascading Ensemble classification framework (MACE) for the automatic annotation of IoT data. MACE is capable of sequentially combining standalone classifiers, enabling it to cope with heterogeneous IoT data and different domains of information (e.g. numerical and textual), which have not been considered previously. MACE incorporates a novel ensemble approach for automatically selecting, sorting, filtering, and assembling classifiers in a way that improves annotation performance. The paper presents extensive experimental evaluations of MACE using public IoT datasets. Results demonstrate that the MACE framework significantly outperforms existing solutions for open IoT data by as much as 10% in classification accuracy.

Discovering NDM-1 inhibitors using molecular substructure embeddings representations.

NDM-1 (New-Delhi-Metallo-β-lactamase-1) is an enzyme developed by bacteria that is implicated in bacteria resistance to almost all known antibiotics. In this study, we deliver a new, curated NDM-1 bioactivities database, along with a set of unifying rules for managing different activity properties and inconsistencies. We define the activity classification problem in terms of Multiple Instance Learning, employing embeddings corresponding to molecular substructures and present an ensemble ranking and classification framework, relaying on a k-fold Cross Validation method employing a per fold hyper-parameter optimization procedure, showing promising generalization ability. The MIL paradigm displayed an improvement up to 45.7 %, in terms of Balanced Accuracy, in comparison to the classical Machine Learning paradigm. Moreover, we investigate different compact molecular representations, based on atomic or bi-atomic substructures. Finally, we scanned the Drugbank for strongly active compounds and we present the top-15 ranked compounds.

A hybrid learning framework for multi-modal facial prediction and recognition using improvised non-linear SVM classifier

Multi-modal classification plays a vital role for the real-time applications since most of the conventional models are independent of homogeneous features with single classifier. In addition, traditional directional gradient descriptors are difficult to find the heterogeneous features on different biometric classification. Feature extraction, segmentation, and multi-modal classification are the essential key factors that improve the true positive rate, error rate, and false positive rate of human-based recognition systems. In this work, a hybrid ensemble-based feature selection ranking measure, hybrid segmentation, and ensemble multi-class multi-modal classification framework on different biometric features is designed. Experimental results show that the proposed multi-class multi-modal ensemble classification framework has better optimization in terms of false positive rate, error rate, and precision than the conventional homogeneous local gradient feature extraction-based classification models on different features.

A clinical decision support system for heart disease prediction with ensemble two-fold classification framework

Cardiovascular disease (CVD) is a severe public health concern globally. Early and accurate CVD diagnosis is a difficult task but a necessary endeavour required to prevent further damage and protect patients’ lives. Machine Learning (ML)-based Clinical Decision Support Systems (CDSS) have the potential to assist healthcare providers in making accurate CVD diagnoses and treatments. Clinical data usually contains missing values (MVs); hence, the incorporated imputation techniques for ML have become a critical consideration when working with real-world medical datasets. Furthermore, removing instances with MVs will lead to essential data loss and produce incorrect results. To overcome these issues, this paper proposes an efficient and reliable CDSS with Ensemble Two-Fold Classification (ETC) framework for classifying heart diseases. The effectiveness of the proposed ETC framework using different supervised ML algorithms is evaluated with four distinct imputation methods for handling MVs over the standard benchmark dataset, viz., the University of California, Irwin (UCI). Experimental results show that our proposed ETC framework with the k-Nearest Neighbors(k-NN) imputation method achieves better classification accuracy of 0.9999 and a lesser error rate of 0.0989 compared to other imputation methods and classifiers with similar execution times.

SSEC: Semantic Segmentation and Ensemble Classification Framework for Static Hand Gesture Recognition using RGB-D Data

Hand Gesture Recognition (HGR) refers to identifying various hand postures used in Sign Language Recognition (SLR) and Human Computer Interaction (HCI) applications. Complex background in uncontrolled environmental condition is the major challenging issue which impacts the recognition accuracy of HGR system. This can be effectively addressed by discarding the background using suitable semantic segmentation method, where it predicts the hand region pixels into foreground and rest of the pixels into background. In this paper, we have analyzed and evaluated well known semantic segmentation architectures for hand region segmentation using both RGB and depth data. Further, ensemble of segmented RGB and depth stream is used for hand gesture classification through probability score fusion. Experimental results shows that the proposed novel framework of Semantic Segmentation and Ensemble Classification (SSEC) is suitable for static hand gesture recognition and achieved F1-score of 88.91% on OUHANDS test dataset.

ScWECTA: A weighted ensemble classification framework for cell type assignment based on single cell transcriptome

Rapid advances in single-cell transcriptome analysis provide deeper insights into the study of tissue heterogeneity at the cellular level. Unsupervised clustering can identify potential cell populations in single-cell RNA-sequencing (scRNA-seq) data, but fail to further determine the identity of each cell. Existing automatic annotation methods using scRNA-seq data based on machine learning mainly use single feature set and single classifier. In view of this, we propose a Weighted Ensemble classification framework for Cell Type Annotation, named scWECTA, which improves the accuracy of cell type identification. scWECTA uses five informative gene sets and integrates five classifiers based on soft weighted ensemble framework. And the ensemble weights are inferred through the constrained non-negative least squares. Validated on multiple pairs of scRNA-seq datasets, scWECTA is able to accurately annotate scRNA-seq data across platforms and across tissues, especially for imbalanced data containing rare cell types. Moreover, scWECTA outperforms other comparable methods in balancing the prediction accuracy of common cell types and the unassigned rate of non-common cell types at the same time. The source code of scWECTA is freely available at https://github.com/ttren-sc/scWECTA.

Deep Ensemble Model for Detecting Attacks in Industrial IoT

In this research work, a novel IIoT attack detection framework is designed by following four major phases: pre-processing, imbalance processing, feature extraction, and attack detection. The attack detection is carried out using the projected ensemble classification framework. The projected ensemble classification framework encapsulates the recurrent neural network, CNN, and optimized bi-directional long short-term memory (BI-LSTM). The RNN and CNN in the ensemble classification framework is trained with the extracted features. The outcome acquired from RNN and CNN is utilized for training the optimized BI-LSTM model. The final outcome regarding the presence/absence of attacks in the industrial IoT is portrayed by the optimized BI-LSTM model. Therefore, the weight of BI-LSTM model is fine-tuned using the newly projected hybrid optimization model referred as cat mouse updated slime mould algorithm (CMUSMA). The projected hybrids the concepts of both the standard slime mould algorithm (SMA) and cat and mouse-based optimizer(CMBO), respectively.

How far does the predictive decision impact the software project? The cost, service time, and failure analysis from a cross-project defect prediction model

Context:Cross-project defect prediction (CPDP) models are being developed to optimise the testing resources. Objectives:Proposing an ensemble classification framework for CPDP as many existing models are lacking with better performances and analysing the main objectives of CPDP from the outcomes of the proposed classification framework. Method:For the classification task, we propose a bootstrap aggregation based hybrid-inducer ensemble learning (HIEL) technique that uses probabilistic weighted majority voting (PWMV) strategy. To know the impact of HIEL on the software project, we propose three project-specific performance measures such as percent of perfect cleans (PPC), percent of non-perfect cleans (PNPC), and false omission rate (FOR) from the predictions to calculate the amount of saved cost, remaining service time, and percent of the failures in the target project. Results:On many target projects from PROMISE, NASA, and AEEEM repositories, the proposed model outperformed recent works such as TDS, TCA+, HYDRA, TPTL, and CODEP in terms of F-measure. In terms of AUC, the TCA+ and HYDRA models stand as strong competitors to the HIEL model. Conclusion:For better predictions, we recommend ensemble learning approaches for the CPDP models. And, to estimate the benefits from the SDP models, we recommend the above project-specific performance measures.

An ensemble contrastive classification framework for imbalanced learning with sample-neighbors pair construction

While existing imbalanced classification methods have made great progress, there are still some challenges in the current imbalanced learning field: (1) How to achieve the balance between classes without introducing noise and losing information; (2) How to mine the differences between classes from datasets with a relatively small number of positive samples; (3) How to learn the distribution differences in overlapping areas efficiently. To address these problems, this paper proposes an ensemble contrastive classification framework with sample-neighbors pair construction. The traditional pointwise imbalanced classification is redefined as a pairwise label matching task between the sample to be classified (Target Sample, TS) and a group of neighbor samples (Contrastive Sample Group, CSG). For any TS, we can obtain multiple CSGs containing same/different class with the TS, and combine the TS with different CSGs into sample-neighbors pairs as positive/negative samples in the new task. In this way, the balance of classes and multiplied increase of sample scale are achieved without introducing any noise. Based on the obtained rich data, a robust classifier can be trained to mine the distribution differences in overlapping areas through the contrastive learning between TS and its CSGs. For a given test sample, we can obtain abundant sample-neighbors pairs and their corresponding classification results. Its label can be obtained by result integration and reverse reasoning. Extensive experimental evaluations on 48 KEEL and UCI public datasets show that the proposed method outperforms the existing state-of-the-art imbalanced classification methods in terms of F1-measure and G-mean.

ScIAE: an integrative autoencoder-based ensemble classification framework for single-cell RNA-seq data.

Single-cell RNA sequencing (scRNA-seq) allows quantitative analysis of gene expression at the level of single cells, beneficial to study cell heterogeneity. The recognition of cell types facilitates the construction of cell atlas in complex tissues or organisms, which is the basis of almost all downstream scRNA-seq data analyses. Using disease-related scRNA-seq data to perform the prediction of disease status can facilitate the specific diagnosis and personalized treatment of disease. Since single-cell gene expression data are high-dimensional and sparse with dropouts, we propose scIAE, an integrative autoencoder-based ensemble classification framework, to firstly perform multiple random projections and apply integrative and devisable autoencoders (integrating stacked, denoising and sparse autoencoders) to obtain compressed representations. Then base classifiers are built on the lower-dimensional representations and the predictions from all base models are integrated. The comparison of scIAE and common feature extraction methods shows that scIAE is effective and robust, independent of the choice of dimension, which is beneficial to subsequent cell classification. By testing scIAE on different types of data and comparing it with existing general and single-cell-specific classification methods, it is proven that scIAE has a great classification power in cell type annotation intradataset, across batches, across platforms and across species, and also disease status prediction. The architecture of scIAE is flexible and devisable, and it is available at https://github.com/JGuan-lab/scIAE.

Sensory motor imagery EEG classification based on non-dyadic wavelets using dynamic weighted majority ensemble classification

The classification accuracy has become a significant challenge and an important task in sensory motor imagery (SMI) electroencephalogram (EEG) based Brain Computer interface (BCI) system. This paper compares ensemble classification framework with individual classifiers. The main objective is to reduce the inference of non-stationary and transient information and improves the classification decision in BCI system. The framework comprises the three phases as follows: (1) the EEG signal first decomposes into triadic frequency bands: low pass band, band pass filter and high pass filter to localize α, β and high γ frequency bands within the EEG signals, (2) Then, Common spatial pattern (CSP) algorithm has been applied on the extracted frequencies in phase I to heave out the important features of EEG signal, (3) Further, an existing Dynamic Weighted Majiority (DWM) ensemble classification algorithm has been implemented using features extracted in phase II, for final class label decision. J48, Naive Bayes, Support Vector Machine, and K-Nearest Neighbor classifiers used as base classifiers for making a diverse ensemble of classifiers. A comparative study between individual classifiers and ensemble framework has been included in the paper. Experimental evaluation and assessment of the performance of the proposed model is done on the publically available datasets: BCI Competition IV dataset IIa and BCI Competition III dataset IVa. The ensemble based learning method gave the highest accuracy among all. The average sensitivity, specificity, and accuracy of 85.4%, 86.5%, and 85.6% were achieved with a kappa value of 0.59 using DWM classification.

Incorporating Artificial Fish Swarm in Ensemble Classification Framework for Recurrence Prediction of Cervical Cancer

IoT has facilitated predominant advancements in cancer research by incorporating Artificial intelligence (AI) that enables human decision-makers to achieve a better decision. Recently, Least Absolute Shrinkage and Selection Operator (LASSO) classifier has launched in predicting recurrence cancer genes in the cervix. At the initial phase, the recurrence gene expression of lncRNA is collected from Geo Datasets. Secondly, data imputation, accomplished with Mode and Mean Missing method (MMM-DI). Thirdly, feature selection is compassed using Hilbert-Schmidt independence criterion with Diversity based Artificial Fish Swarm (HSDAFS). In the HSDA.FS algorithm, the diversity parameter is added based on the gene value, and their risk score of the lncRNAs is computed using the Artificial intelligence (AI) technique. Finally, recurrence prediction, an ENSemble Classification Framework (ENSCF), is proposed based on recurrent neural networks. The prognostic factor is computed with a risk score of nine lncRNA signatures for 300 samples taken from GSE44001. The Chi-Square method has been used to obtain statistical results. The survival of the patient with recurrence cervical cancer is shown using the proposed model.

SmartPrognosis: Automatic ensemble classification for quantitative EEG analysis in patients resuscitated from cardiac arrest

Understanding a patient’s current status, anticipated clinical course, and likely outcomes can be critical to the practice of medicine. Among patients who are comatose after resuscitation from cardiac arrest, identifying those with potential for awakening and favorable recovery is challenging. Currently, this task is accomplished through the acquisition of one or more diagnostic modalities that aim to assess brain function, with expert interpretation of the test results. This approach is subjective, imprecise, and not scalable. We propose an automatic ensemble classification framework, named SmartPrognosis, to identify comatose post-arrest patients with no recovery potential. SmartPrognosis automatically generates and assembles candidate machine learning pipelines with high sensitivity predicting poor outcomes at a fixed near-zero error rate of misclassifying patients with good outcomes. We demonstrate the effectiveness of SmartPrognosis on real patient data, showing that it over-performs commonly used alternative approaches on all evaluation metrics.

An Enhanced Sentiment Analysis Framework Based on Pre-Trained Word Embedding

Sentiment analysis (SA) is a technique that lets people in different fields such as business, economy, research, government, and politics to know about people’s opinions, which greatly affects the process of decision-making. SA techniques are classified into: lexicon-based techniques, machine learning techniques, and a hybrid between both approaches. Each approach has its limitations and drawbacks, the machine learning approach depends on manual feature extraction, lexicon-based approach relies on sentiment lexicons that are usually unscalable, unreliable, and manually annotated by human experts. Nowadays, word-embedding techniques have been commonly used in SA classification. Currently, Word2Vec and GloVe are some of the most accurate and usable word embedding techniques, which can transform words into meaningful semantic vectors. However, these techniques ignore sentiment information of texts and require a huge corpus of texts for training and generating accurate vectors, which are used as inputs of deep learning models. In this paper, we propose an enhanced ensemble classifier framework. Our framework is based on our previously published lexicon-based method, bag-of-words, and pre-trained word embedding, first the sentence is preprocessed by removing stop-words, POS tagging, stemming and lemmatization, shortening exaggerated word. Second, the processed sentence is passed to three modules, our previous lexicon-based method (Sum Votes), bag-of-words module and semantic module (Word2Vec and Glove) and produced feature vectors. Finally, the previous features vectors are fed into 11 different classifiers. The proposed framework is tested and evaluated over four datasets with five different lexicons, the experiment results show that our proposed model outperforms the previous lexicon based and the machine learning methods individually.