Ensemble Classification System Research Articles

Design of a brain-machine interface for reducing false activations of a lower-limb exoskeleton based on error related potential

Background and objectiveBrain-Machine Interfaces (BMIs) based on a motor imagination paradigm provide an intuitive approach for the exoskeleton control during gait. However, their clinical applicability remains difficulted by accuracy limitations and sensitivity to false activations. A proposed improvement involves integrating the BMI with methods based on detecting Error Related Potentials (ErrP) to self-tune erroneous commands and enhance not only the system accuracy, but also its usability. The aim of the current research is to characterize the ErrP at the beginning of the gait with a lower limb exoskeleton to reduce the false starts in the BMI system. Furthermore, this study is valuable for determining which type of feedback, Tactile, Visual, or Visuo-Tactile, achieves the best performance in evoking and detecting the ErrP. MethodsThe initial phase of the research concentrates on detecting ErrP at the beginning of gait to improve the efficiency of an asynchronous BMI based on motor imagery (BMI-MI) to control a lower limb exoskeleton. Initially, an experimental protocol is designed to evoke ErrP at the start of gait, employing three different stimuli: Tactile, Visual, and Visuo-Tactile. An iterative selection process is then utilized to characterize ErrP in both time and frequency domains and fine-tune various parameters, including electrode distribution, feature combinations, and classifiers. A generic classifier with 6 subjects is employed to configure an ensemble classification system for detecting ErrP and reducing the false starts. ResultsThe ensembled system configured with the selected parameters yields an average correction of false starts of 72.60 % ± 10.23, highlighting its corrective efficacy. Tactile feedback emerges as the most effective stimulus, outperforming Visual and Visuo-Tactile feedback in both training types. ConclusionsThe results suggest promising prospects for reducing the false starts when integrating ErrP with BMI-MI, employing Tactile feedback. Consequently, the security of the system is enhanced. Subsequent, further research efforts will focus on detecting error potential during movement for gait stop, in order to limit undesired stops.

Genetic Algorithm-Based Multitier Ensemble Classifier for Diagnosis of Heart Disease

Designing a hybrid or ensemble data mining system appropriate to the application is a research challenge. Heart disease is a life threatening disease that need to be recognized correctly in the starting stage before it becomes more complex. Using artificial intelligence techniques in a hybrid and ensemble architecture can support the prediction of heart disease more effectively based on the given sample cases. This paper proposes a classification system called genetic algorithm-based ensemble classification system (GA-ECS) for the identification of heart disease. As feature selection is the crucial step before applying the data mining techniques, the genetic algorithm is used in GA-ECS to identify the best features in a given dataset. The Cleveland heart disease dataset is used for testing GA-ECS. The performance of GA-ECS is compared with different machine learning classifiers for the prediction of heart disease. GA-ECS showed a promising outcome with an accuracy of 90% for the diagnosis of heart disease.

Mangrove forest mapping from object-oriented multi-feature ensemble classification using Sentinel-2 images

Accurate mapping of mangrove forests is crucial for understanding their ecosystem function and developing effective management policies. However, the absence of an operational multi-feature fusion approach and an ensemble classification system restricts the achievement of this goal. This study aims to develop an object-oriented multi-feature ensemble classification scheme (OMEC). First, an enhanced mangrove spectral index (EMSI) is established by analyzing the spectral reflectance differences between mangrove forests and other land cover types. Sentinel-2 images are segmented into objects using the multi-resolution segmentation method. Then, spectral, textural, and geometric features are extracted, and these features (including EMSI) are inputted into the nearest neighbor classifier to implement mangrove classification. The experiment was conducted in three typical mangrove areas in China using Sentinle-2 images. The results demonstrate that EMSI exhibits good spectral separability for mangroves and performs well in the ensemble classification scheme. The overall accuracy of mangrove classification exceeds 90%, with a Kappa coefficient greater than 0.88. The object-oriented multi-feature ensemble classification scheme significantly improves accuracy and exhibits excellent performance. The method enhances the accuracy of mangrove classification, enriches the approach to mangrove remote sensing interpretation, and offers data support and scientific references for the restoration, management, and protection of coastal wetlands.

Construction of Exosome SORL1 Detection Platform Based on 3D Porous Microfluidic Chip and its Application in Early Diagnosis of Colorectal Cancer.

Exosomes are promising new biomarkers for colorectal cancer (CRC) diagnosis, due to their rich biological fingerprints and high level of stability. However, the accurate detection of exosomes with specific surface receptors is limited to clinical application. Herein, an exosome enrichment platform on a 3D porous sponge microfluidic chip is constructed and the exosome capture efficiency of this chip is ≈90%. Also, deep mass spectrometry analysis followed by multi-level expression screenings revealed a CRC-specific exosome membrane protein (SORL1). A method of SORL1 detection by specific quantum dot labeling is further designed and the ensemble classification system is established by extracting features from 64-patched fluorescence images. Importantly, the area under the curve (AUC) using this system is 0.99, which is significantly higher (p< 0.001) than that using a conventional biomarker (carcinoembryonicantigen(CEA), AUC of 0.71). The above system showed similar diagnostic performance, dealing with early-stage CRC, young CRC, and CEA-negative CRC patients.

Fractal-Based Ensemble Classification System for Hyperspectral Images

Fractal-Based Ensemble Classification System for Hyperspectral Images

A discrimination model for differentiation of renal cell carcinoma from renal angiomyolipoma without visible fat: based on hierarchical fusion framework of multi-classifier

To investigate the capabilities of classification models based on hierarchical fusion framework of multi-classifier using a random projection strategy for differentiation of renal cell carcinoma (RCC) from small renal angiomyolipoma (< 4 cm) without visible fat (AMLwvf). We retrospectively collected the clinical data from 163 patients with pathologically proven small renal mass, including 118 with RCC and 45 with AMLwvf.Target region of interest (ROI) delineation was performed on an unenhanced phase (UP) CT image slice displaying the largest lesion area.The radiomics features were used to establish a hierarchical fusion method.On the projection-based level, the homogeneous classifiers were fused, and the fusion results were further fused at the classifier-based level to construct a multi-classifier fusion system based on random projection for differentiation of AMLwvf and RCC.The discriminative capability of this model was quantitatively evaluated using 5-fold cross validation and 4 evaluation indexes[specificity, sensitivity, accuracy and area under ROC curve (AUC)].We quantitatively compared this multi-classifier fusion framework against different classification models using a single classifier and several multi-classifier ensemble models. When the projection number was set at 10, the proposed hierarchical fusion differentiation framework achieved the best results on all the evaluation measurements.At the optimal projection number of 10, the specificity, sensitivity, average accuracy and AUC of the multi-classifier ensemble classification system for differentiation between AMLwvf and RCC were 0.853, 0.693, 0.809 and 0.870, respectively. The proposed model constructed based on a multi-classifier fusion system using random projection shows better performance to differentiate RCC from AMLwvf than the AMLwvf and RCC discrimination models based on a single classification algorithm and the currently available benchmark ensemble methods.

Uncertainty-driven ensembles of multi-scale deep architectures for image classification

The use of automatic systems for medical image classification has revolutionized the diagnosis of a high number of diseases. These alternatives, which are usually based on artificial intelligence (AI), provide a helpful tool for clinicians, eliminating the inter and intra-observer variability that the diagnostic process entails. Convolutional Neural Network (CNNs) have proved to be an excellent option for this purpose, demonstrating a large performance in a wide range of contexts. However, it is also extremely important to quantify the reliability of the model’s predictions in order to guarantee the confidence in the classification. In this work, we propose a multi-level ensemble classification system based on a Bayesian Deep Learning approach in order to maximize performance while providing the uncertainty of each classification decision. This tool combines the information extracted from different architectures by weighting their results according to the uncertainty of their predictions. Performance is evaluated in a wide range of real scenarios: in the first one, the aim is to differentiate between different pulmonary pathologies: controls vs bacterial pneumonia vs viral pneumonia. A two-level decision tree is employed to divide the 3-class classification into two binary classifications, yielding an accuracy of 98.19%. In the second context, performance is assessed for the diagnosis of Parkinson’s disease, leading to an accuracy of 95.31%. The reduced preprocessing needed for obtaining this high performance, in addition to the information provided about the reliability of the predictions evidence the applicability of the system to be used as an aid for clinicians.

Development of a Flexible Ensemble Classification System for Microscopy

An abstract is not available for this content so a preview has been provided. As you have access to this content, a full PDF is available via the ‘Save PDF’ action button.

Ensemble learning-based abnormality diagnosis in wrist skeleton radiographs using densenet variants voting

Almost one out of five people, including children, suffers from musculoskeletal disorders. It is the second leading cause of disability worldwide. It affects the musculoskeletal system’s major areas, represented by the shoulder, forearm, and wrist. It causes severe pain, joint noises, and disability. To detect the abnormality, the radiologist analyzes the patient’s anatomy through X-rays of different views and projections. To automatically diagnose the abnormality in the musculoskeletal system is a challenging task. Previously, various researchers detected the abnormality in the musculoskeletal system from radiographic images by using several deep learning techniques. They used a capsule network, 169-layer convolutional neural network, and group normalized convolutional neural network in musculoskeletal abnormality detection. However, to propose methods for improving abnormality detection, further work needs to be done because the accuracy of the conventional methods is far away from 90%. This paper presents an ensemble learning-based classification system for detecting abnormality in wrist radiographs. Tags in radiographs may result in learning noisy features hence reducing the performance. Therefore, tags are segmented and removed using UNet trained on the annotated ground truths. Segmented images are then used for voting-based diagnosis. The simulation results show that the proposed methodology improves testing accuracy by 1.5%-4.5% compared to the available wrist abnormality detection methods. The proposed methodology can be used for any kind of musculoskeletal abnormality detection.

ECG-BiCoNet: An ECG-based pipeline for COVID-19 diagnosis using Bi-Layers of deep features integration

The accurate and speedy detection of COVID-19 is essential to avert the fast propagation of the virus, alleviate lockdown constraints and diminish the burden on health organizations. Currently, the methods used to diagnose COVID-19 have several limitations, thus new techniques need to be investigated to improve the diagnosis and overcome these limitations. Taking into consideration the great benefits of electrocardiogram (ECG) applications, this paper proposes a new pipeline called ECG-BiCoNet to investigate the potential of using ECG data for diagnosing COVID-19. ECG-BiCoNet employs five deep learning models of distinct structural design. ECG-BiCoNet extracts two levels of features from two different layers of each deep learning technique. Features mined from higher layers are fused using discrete wavelet transform and then integrated with lower-layers features. Afterward, a feature selection approach is utilized. Finally, an ensemble classification system is built to merge predictions of three machine learning classifiers. ECG-BiCoNet accomplishes two classification categories, binary and multiclass. The results of ECG-BiCoNet present a promising COVID-19 performance with an accuracy of 98.8% and 91.73% for binary and multiclass classification categories. These results verify that ECG data may be used to diagnose COVID-19 which can help clinicians in the automatic diagnosis and overcome limitations of manual diagnosis.

A new method for ensemble combination based on adaptive decision making

Ensemble classifiers methods are considered as one of the strongest classification approaches in different applications. A key factor in creating a successful ensemble is to design an optimal combiner that minimizes classification errors. Unfortunately, finding the best fusion methodology that maximizes ensemble performance is a challenge. Most popular ensemble classifiers systems use fixed fusion methods such as average and majority voting. For a given application, there is no clear understanding of why a fusion method might work better than other methods. In this work, decision-making based on Adaptive Power Mean Combiner (APMC) is investigated. A mathematical and experimental framework for ensemble classifier systems based on APMC is proposed. The theoretical results explain why a given algebraic combiner such as average, product, maximum, minimum, and others works best for a given dataset while providing poor results on other datasets. It was shown that selecting an optimal fusion method is strongly dependent on the statistics of base learners’ outputs. Theoretical results for a given ensemble condition show that an improvement in classification accuracy of ∼8% compared to the average fusion method is achieved. Guided by theoretical results, we propose an experimental ensemble-learning algorithm based on surrogate optimization method for searching fusion methods for a given ensemble condition. The performance trend predicted by the mathematical model is validated through several datasets and compared against different fusion methods. Experimental results show the proposed algorithm achieves notable classification results compared to several fusion methods.

Learning to Learn: Algorithmic Inspirations from Human Problem Solving

We harness the ability of people to perceive and interact with visual patterns in order to enhance the performance of a machine learning method. We show how we can collect evidence about how people optimize the parameters of an ensemble classification system using a tool that provides a visualization of misclassification costs. Then, we use these observations about human attempts to minimize cost in order to extend the performance of a state-of-the-art ensemble classification system. The study highlights opportunities for learning from evidence collected about human problem solving to refine and extend automated learning and inference.

Detection of counterfeit coins based on 3D height-map image analysis

Detecting a counterfeit coin using 2D image processing is nearly impossible in some cases, especially when the coin is damaged, corroded or worn out. Edge detection is one of the most widely used techniques to extract features from 2D images. However, in 2D images, the height information is missing, losing the hidden characteristics. In this paper, we propose a 3D approach to detect and analyze the precipice borders from the coin surface and extract significant features to train an ensemble classification system. To extract the features, we also propose Binned Borders in Spherical Coordinates (BBSC) to analyze different parts of precipice borders at different polar and azimuthal angles. The proposed method is robust even against degradation which appears on shiny coins after 3D scanning. Therefore, there is no need to restore the degraded images before the feature extraction process. Here, the system has been trained and tested with four types of Danish and two types of Chinese coins. We take advantage of stack generalization to classify the coins and add the reject option to increase the reliability of the system. The results illustrate that the proposed method outperforms other counterfeit coin detectors. The accuracy obtained by testing Danish 1990, 1991, 1996, and 2008 datasets are 98.6%, 98.0%, 99.8%, and 99.9% respectively. In addition, results for half Yuan Chinese 1942 and one Yuan Chinese 1997 were 95.5% and 92.2% respectively.

On the Potential of Taxonomic Graphs to Improve Applicability and Performance for the Classification of Biomedical Patents

A core task in technology management in biomedical engineering and beyond is the classification of patents into domain-specific categories, increasingly automated by machine learning, with the fuzzy language of patents causing particular problems. Striving for higher classification performance, increasingly complex models have been developed, based not only on text but also on a wealth of distinct (meta) data and methods. However, this makes it difficult to access and integrate data and to fuse distinct predictions. Although the already established Cooperate Patent Classification (CPC) offers a plethora of information, it is rarely used in automated patent categorization. Thus, we combine taxonomic and textual information to an ensemble classification system comparing stacking and fixed combination rules as fusion methods. Various classifiers are trained on title/abstract and on both the CPC and IPC (International Patent Classification) assignments of 1230 patents covering six categories of future biomedical innovation. The taxonomies are modeled as tree graphs, parsed and transformed by Dissimilarity Space Embedding (DSE) to real-valued vectors. The classifier ensemble tops the basic performance by nearly 10 points to F1 = 78.7% when stacked with a feed-forward Artificial Neural Network (ANN). Taxonomic base classifiers perform nearly as well as the text-based learners. Moreover, an ensemble only of CPC and IPC learners reaches F1 = 71.2% as fully language independent and straightforward approach of established algorithms and readily available integrated data enabling new possibilities for technology management.

Post classification change detection based on feature-based ensemble classifiers

Change detection is a challenging task in the field of remote sensing. Mainly, the change map is used for disaster assessment, monitoring deforestation and urban studies. In this paper, we present a novel method for post classification change detection. Google Earth images of 2011 and 2016 of Bangalore East are used for the study. Multiple features such as texture features, morphological features are extracted using grey level co-occurrence matrix (GLCM) and morphological operations respectively. Linear discriminant analysis (LDA) is used to reduce the dimension of the selected features for the training set. The proposed ensemble classifier system (ECS) exploits K-nearest neighbour (KNN), support vector machine (SVM) and maximum likelihood classifier (MLC). The proposed method adopts the subsample kernel-based subtraction technique to find the difference image; this method greatly reduces data complexity compared to per-pixel-based image subtraction method.

A new synergistic approach for crop discrimination in a semi-arid region using Sentinel-2 time series and the multiple combination of machine learning classifiers

Accurate monitoring of agricultural lands and crop types is a crucial tool for sustainable food production. Therefore, to provide reliable and updated crop maps, the improvement of satellite image classification approaches is essential. In this context, machine learning algorithms present a potential tool for efficient and effective classification of remotely sensed data. The main strengths of machine learning algorithms are the capacity to handle data of high dimensionality, and mapping classes characterized by strong complex dynamics. The main objective of this work was to develop a new synergistic approach for crop discrimination in the semi-arid region of Chichaoua province, located in the Marrakesh-Safi region, Morocco, using high spatio-temporal resolution imagery and a multiple combination of machine learning classifiers. This approach was developed based on 10m spatial resolution open access Sentinel-2 (S2) images and machine learning algorithms. The atmospherically corrected S2 images were accessed through the Theia Land Data Center. Reference dataset was collected from a field survey carried out during the 2018 agricultural season in order to train the classifiers. Artificial Neural Networks, Support Vector Machine, K-Nearest Neighbors, Bagged Trees, Naive Bayes, Discriminant Analysis and Decision Trees classifiers were trained over the study area and the accuracy metrics, mainly Overall Accuracy (OA) and Kappa coefficient (K), were assessed. The trained models were single classifiers to build the ensemble classifier system. The obtained results showed high OA and K values up to 96% and 0.95 respectively, achieved by the developed approach. Therefore, based on these results, the approach we developed using the combination of multiple classifiers has a significant impact on crop classification quality.

Transferring CNN Ensemble for Hyperspectral Image Classification

In recent years, deep convolutional neural networks (CNNs) have been widely investigated for hyperspectral image (HSI) classification. The CNN-based HSI classifiers obtained good performance under the condition of sufficient training samples. In order to address the problem of limited training samples, in this letter, transfer learning is combined with CNN to address the issue of HSI classification. Pretrained models on large-scale data sets (e.g., ImageNet) can extract the general and discriminative features. Due to the fact that the extracted low-level and mid-level features can be reused for the HSI feature extraction, the CNN-based methods usually obtain good classification performance with insufficient training samples. The ImageNet data set has three channels, while the HSI data set contains hundreds of channels. Therefore, three channels of the HSI are randomly selected to formulate a transferring CNN. Then, several transferring CNNs are combined to establish an ensemble classification system with diversity. Moreover, an improved label smoothing technique is proposed to further improve the classification accuracy of the HSI. Experimental results on two popular hyperspectral data sets [i.e., Indian Pines and Kennedy Space Center (KSC)] show that the transferring CNN ensemble obtains good classification performance compared to the state-of-the-art methods.

Ensemble Malware Classification System Using Deep Neural Networks

With the advancement of technology, there is a growing need of classifying malware programs that could potentially harm any computer system and/or smaller devices. In this research, an ensemble classification system comprising convolutional and recurrent neural networks is proposed to distinguish malware programs. Microsoft’s Malware Classification Challenge (BIG 2015) dataset with nine distinct classes is utilized for this study. This dataset contains an assembly file and a compiled file for each malware program. Compiled files are visualized as images and are classified using Convolutional Neural Networks (CNNs). Assembly files consist of machine language opcodes that are distinguished among classes using Long Short-Term Memory (LSTM) networks after converting them into sequences. In addition, features are extracted from these architectures (CNNs and LSTM) and are classified using a support vector machine or logistic regression. An accuracy of 97.2% is achieved using LSTM network for distinguishing assembly files, 99.4% using CNN architecture for classifying compiled files and an overall accuracy of 99.8% using the proposed ensemble approach thereby setting a new benchmark. An independent and automated classification system for assembly and/or compiled files provides the luxury to anti-malware industry experts to choose the type of system depending on their available computational resources.

Performance assessment of ensemble learning systems in financial data classification

SummaryFinancial data classification plays an important role in investment and banking industry with the purpose to control default risk, improve cash and select the best customers. Ensemble learning and classification systems are becoming gradually more applied to classify financial data where outputs from different classification systems are combined. The objective of this research is to assess the relative performance of existing state‐of‐the‐art ensemble learning and classification systems with applications to corporate bankruptcy prediction and credit scoring. The considered ensemble systems include AdaBoost, LogitBoost, RUSBoost, subspace, and bagging ensemble system. The experimental results from three datasets: one is composed of quantitative attributes, one encompasses qualitative data, and another one combines both quantitative and qualitative attributes. By using ten‐fold cross‐validation method, the experimental results show that AdaBoost is effective in terms of low classification error, limited complexity, and short time processing of the data. In addition, the experimental results show that ensemble classification systems outperform existing models that were recently validated on the same databases. Therefore, ensemble classification system can be employed to increase the reliability and consistency of financial data classification task.

A Genotype-Based Ensemble Classifier System for Non-Small-Cell Lung Cancer

The heterogeneity of cancer reflects the complexity of genetic mutations. Dissecting the heterogeneity plays an important role in the field of biomarker discovery, targeted therapy and drug designing. As it is time-consuming to identify new biomarkers in biological experiments, various machine learning methods have been developed. However, the current methods are limited because they ignore that patients may correspond to different disease-causing genotypes. In this article, a genotype-based ensemble classifier system (GECS) is proposed which aims to explain pathologies of NSCLC from the view of genotypes and identify the genetic subtypes of tumors. The core strategy of GECS is to construct multiple independent classifiers following the principle that one classifier is constructed based on one genotype of NSCLC. The analysis of synthetic data and three microarray datasets indicated that the proposed method outperforms existing approaches in the identification of genetic subtypes of tumors. The GECS method provides a useful tool for molecular pathology researches for dissecting the heterogeneity of cancer.