Support Vector Machine Ensemble Research Articles

Dynamic correction of soft measurement model for evaporation process parameters based on ARMA.

To address the significant soft measurement errors in traditional static models for evaporation process parameters, which are characterized by continuity and cumulativity, this paper proposes a dynamic correction method for soft measurement models of evaporation process parameters based on the autoregressive moving-average model (ARMA). Initially, the Powell's directional evolution (Powell-DE) algorithm is utilized to identify the autoregressive order and moving average order of the ARMA model. Subsequently, the prediction error of a mechanism-reduced robust least squares support vector machine ensemble model is utilized as input. An error time series prediction model, which compensates for the errors in the autoregressive moving average model, is then applied for dynamic estimation of the prediction error. Finally, an integration strategy using the entropy method is employed to combine the static soft measurement model, based on the mechanism-reduced robust least squares support vector machine, with the dynamic correction soft measurement model, which is based on the error time series compensation of the ARMA model. The new model is analyzed and validated using production data from an alumina plant's evaporation process. Compared to traditional models, the new model demonstrates significantly improved prediction accuracy and is capable of dynamic prediction of evaporation process parameters.

A Spectral–Spatial Feature Rotation-Based Ensemble Method for Imbalanced Hyperspectral Image Classification

Hyperspectral image classification has two key characteristics: spatial dependency and class imbalance. The synthetic oversampling methods for solving the between-class imbalance has the problem of spatial information loss to some extent. However, the feature space optimization methods for solving data within-class complex distribution problems have not been studied extensively. To remedy such issue, we propose in this paper an improved spectral-spatial feature rotation based SVM ensemble method (SSFRoF). More precisely, in the data preprocessing stage and from the perspective of sample space optimization, we devise a spectral-spatial neighborhood based adaptive synthetic sampling algorithm (SSAS) that focuses adaptively on hard-to-learn samples, with artificial samples interpolated in the spectral-spatial neighborhood to further utilize the spatial information. In feature rotation stage and from the perspective of feature space optimization, we innovatively discover that the class overlapping problem can be handled by combining feature extraction, so as to achieve the goal of reducing the difficulty of imbalanced classification. Besides, the SVM and voting procedure are used in the ensembled classification stage. We then conduct a series of experiments to compare the proposed SSFRoS with several baselines in terms of seven metrics. The results show that SSFRoS has significant advantages in dealing with imbalanced hyperspectral image classification problem. In addition, a detailed ablation study shows that the lowest class recall can be improved by 46.67% and 5.26% on Indian Pines and Pavia University respectively, which demonstrates that using SSAS can enhance the recognition of hard-to-learn classes.

Ensemble Model for Diagnostic Classification of Alzheimer's Disease Based on Brain Anatomical Magnetic Resonance Imaging.

Alzheimer's is one of the fast-growing diseases among people worldwide leading to brain atrophy. Neuroimaging reveals extensive information about the brain's anatomy and enables the identification of diagnostic features. Artificial intelligence (AI) in neuroimaging has the potential to significantly enhance the treatment process for Alzheimer's disease (AD). The objective of this study is two-fold: (1) to compare existing Machine Learning (ML) algorithms for the classification of AD. (2) To propose an effective ensemble-based model for the same and to perform its comparative analysis. In this study, data from the Alzheimer's Diseases Neuroimaging Initiative (ADNI), an online repository, is utilized for experimentation consisting of 2125 neuroimages of Alzheimer's disease (n = 975), mild cognitive impairment (n = 538) and cognitive normal (n = 612). For classification, the framework incorporates a Decision Tree (DT), Random Forest (RF), Naïve Bayes (NB), and K-Nearest Neighbor (K-NN) followed by some variations of Support Vector Machine (SVM), such as SVM (RBF kernel), SVM (Polynomial Kernel), and SVM (Sigmoid kernel), as well as Gradient Boost (GB), Extreme Gradient Boosting (XGB) and Multi-layer Perceptron Neural Network (MLP-NN). Afterwards, an Ensemble Based Generic Kernel is presented where Master-Slave architecture is combined to attain better performance. The proposed model is an ensemble of Extreme Gradient Boosting, Decision Tree and SVM_Polynomial kernel (XGB + DT + SVM). At last, the proposed method is evaluated using cross-validation using statistical techniques along with other ML models. The presented ensemble model (XGB + DT + SVM) outperformed existing state-of-the-art algorithms with an accuracy of 89.77%. The efficiency of all the models was optimized using Grid-based tuning, and the results obtained after such process showed significant improvement. XGB + DT + SVM with optimized parameters outperformed all other models with an efficiency of 95.75%. The implication of the proposed ensemble-based learning approach clearly shows the best results compared to other ML models. This experimental comparative analysis improved understanding of the above-defined methods and enhanced their scope and significance in the early detection of Alzheimer's disease.

Mapping dead understoreyBuxus hyrcana Pojarkusing Sentinel-2 and Sentinel-1 data

AbstractThe Hyrcanian Forests comprise a continuous 800-km belt of mostly deciduous broadleaf forests and are considered as Iran’s most important vegetation region in terms of density, canopy cover and species diversity. One of the few evergreen species of the Hyrcanian Forests is the box tree (Buxus), which is seriously threatened by box blight disease and box tree moth outbreaks. Therefore, information on the spatial distribution of intact and infested box trees is essential for recovery monitoring, control treatment and management. To address this critical knowledge gap, we integrated a genetic algorithm (GA) with a support vector machine (SVM) ensemble classification based on the combination of leaf-off optical Sentinel-2 and radar Sentinel-1 data to map the spatial distribution of box tree mortality. We additionally considered the overstorey species composition to account for a potential impact of overstory stand composition on the spectral signature of understorey defoliation. We consequently defined target classes based on the combination of dominant overstorey trees (using two measures including the relative frequency and the diameter at breast height) and two defoliation levels of box trees (including dead and healthy box trees). Our classification workflow applied a GA to simultaneously derive optimal vegetation indices (VIs) and tuning parameters of the SVM. Then the distribution of box tree defoliation was mapped by an SVM ensemble with bagging using GA-optimized VIs and radar data. The GA results revealed that normalized difference vegetation index, red edge normalized difference vegetation index and green normalized difference vegetation index were appropriate for box tree defoliation mapping. An additional comparison of GA-SVM (using GA-optimized VIs and tuning parameters) with a simple SVM (using all VIs and user-based tuning parameters) showed that our suggested workflow performs notably better than the simple SVM (overall accuracy of 0.79 vs 0.74). Incorporating Sentinel-1 data to GA-SVM, marginally improved the performance of the model (overall accuracy: 0.80). The SVM ensemble model using Sentinel-2 and -1 data yielded high accuracies and low uncertainties in mapping of box tree defoliation. The results showed that infested box trees were mostly located at low elevations, low slope and facing north. We conclude that mortality of evergreen understorey tree species can be mapped with good accuracies using freely available satellite data if a suitable work-flow is applied.

Predicting cardiac arrhythmia on ECG signal using an ensemble of optimal multicore support vector machines

Predicting cardiac arrhythmia on ECG signal using an ensemble of optimal multicore support vector machines

Kernel ensemble support vector machine with integrated loss in shared parameters space

Kernel ensemble support vector machine with integrated loss in shared parameters space

Ensemble learning for the detection of pli-de-passages in the superior temporal sulcus

The surface of the cerebral cortex is very convoluted, with a large number of folds, the cortical sulci. These folds are extremely variable from one individual to another, and this large variability is a problem for many applications in neuroscience and brain imaging. In particular, sulcal geometry (shape) and sulcal topology (branches, number of pieces) are very variable. “Plis de passages” (PPs) or “annectant gyri” can explain part of the topological variability, namely why sulci have a variable number of pieces across subjects. The concept of PPs was first introduced by Gratiolet (1854) to describe transverse gyri that interconnect both sides of a sulcus, that are frequently buried in the depth of sulci, and that are sometimes apparent on the cortical surface, hence seemingly interrupting the course of sulci and separating them in several pieces. Nevertheless, the difficulty of identifying PPs and the lack of systematic methods to automatically detect them has limited their use. However, based on a recent characterization of PPs in the superior temporal sulcus, we present here a method to automatically detect PPs in the superior temporal sulcus. Local morphology within the sulcus is characterized using cortical surface profiling, and the three-dimensional PP recognition problem is performed as a two-dimensional image classification problem with class-imbalance. This is solved by using an ensemble support vector machine model (EnsSVM) with a rebalancing strategy. Cross validation and quantitative experimental results on an external dataset show the effectiveness and robustness of our approach.

SVM ensemble training for imbalanced data classification using multi-objective optimization techniques

One of the main problems with classifier training for imbalanced data is defining the correct learning criterion. On the one hand, we want the minority class to be correctly recognized, and on the other hand, we do not want to make too many mistakes in the majority class. Commonly used metrics focus either on the predictive quality of the distinguished class or propose an aggregation of simple metrics. The aggregate metrics, such as Gmean or AUC, are primarily ambiguous, i.e., they do not indicate the specific values of errors made on the minority or majority class. Additionally, improper use of aggregate metrics results in solutions selected with their help that may favor the majority class. The authors realize that a solution to this problem is using overall risk. However, this requires knowledge of the costs associated with errors made between classes, which is often unavailable. Hence, this paper will propose the semoos algorithm - an approach based on multi-objective optimization that optimizes criteria related to the prediction quality of both minority and majority classes. semoos returns a pool of non-dominated solutions from which the user can choose the model that best suits him. Automatic solution selection formulas with a so-called Pareto front have also been proposed to compare state-of-the-art methods. The proposed approach will train a svm classifier ensemble dedicated to the imbalanced data classification task. The experimental evaluations carried out on a large number of benchmark datasets confirm its usefulness.

Soil Classification by Machine Learning Using a Tunnel Boring Machine’s Operating Parameters

This study predicted soil classification using data gathered during the operation of an earth-pressure-balance-type tunnel boring machine (TBM). The prediction methodology used machine learning to find relationships between the TBM’s operating parameters which are monitored continuously during excavation, and the engineering characteristics of the ground which are only available from prior geotechnical investigation. Classification criteria were set using the No. 200 sieve pass rate and N-value and employed classification algorithms that used data for six operating parameters (penetration rate, thrust force, cutterhead torque, screw torque, screw revolution speed, and earth pressure). The results of the ensemble model (i.e., AdaBoost, gradient boosting, XG boosting, and Light GBM), decision tree, and SVM model were examined. As a result, the decision tree and AdaBoost models showed accuracy values of 0.759 to 0.879 in the first and second classification steps, but with poor precision and recall values of around 0.6. In contrast, the gradient boosting, XG boosting, Light GBM, and support vector models all showed excellent performance, with accuracy values over 0.90, and strong precision and recall values. Comparing the performance and the speed of learning using the same PC found Light GBM which showed both excellent learning performance and speed to be a suitable model for predicting soil classification using TBM operating data. The classification model developed here is expected to help guide excavation in sections of ground that lack prior geotechnical information.

Statistical Safety Factor in Lightning Performance Analysis of Overhead Distribution Lines

This paper introduces a novel machine learning (ML) model for the lightning performance analysis of overhead distribution lines (OHLs), which facilitates a data-centrist and statistical view of the problem. The ML model is a bagging ensemble of support vector machines (SVMs), which introduces two significant features. Firstly, support vectors from the SVMs serve as a scaffolding, and at the same time give rise to the so-called curve of limiting parameters for the line. Secondly, the model itself serves as a foundation for the introduction of the statistical safety factor to the lightning performance analysis of OHLs. Both these aspects bolster an end-to-end statistical approach to the OHL insulation coordination and lightning flashover analysis. Furthermore, the ML paradigm brings the added benefit of learning from a large corpus of data amassed by the lightning location networks and fostering, in the process, a “big data” approach to this important engineering problem. Finally, a relationship between safety factor and risk is elucidated. THe benefits of the proposed approach are demonstrated on a typical medium-voltage OHL.

Multimodal magnetic resonance imaging for Alzheimer's disease diagnosis using hybrid features extraction and ensemble support vector machines

AbstractMagnetic resonance imaging (MRI) is increasingly used in the diagnosis of Alzheimer's disease (AD) in order to identify abnormalities in the brain. Indeed, cortical atrophy, a powerful biomarker for AD, can be detected using structural MRI (sMRI), but it cannot detect impairment in the integrity of the white matter (WM) preceding cortical atrophy. The early detection of these changes is made possible by the novel MRI modality known as diffusion tensor imaging (DTI). In this study, we integrate DTI and sMRI as complementary imaging modalities for the early detection of AD in order to create an effective computer‐assisted diagnosis tool. The fused Bag‐of‐Features (BoF) with Speeded‐Up Robust Features (SURF) and modified AlexNet convolutional neural network (CNN) are utilized to extract local and deep features. This is applied to DTI scalar metrics (fractional anisotropy and diffusivity metric) and segmented gray matter images from T1‐weighted MRI images. Then, the classification of local unimodal and deep multimodal features is first performed using support vector machine (SVM) classifiers. Then, the majority voting technique is adopted to predict the final decision from the ensemble SVMs. The study is directed toward the classification of AD versus mild cognitive impairment (MCI) versus cognitively normal (CN) subjects. Our proposed method achieved an accuracy of 98.42% and demonstrated the robustness of multimodality imaging fusion.

MRPO-Deep maxout: Manta ray political optimization based Deep maxout network for big data intrusion detection using spark architecture

An intrusion detection system (IDS) is used to examine as well as analyze data for detecting intrusions in a network or system. This paper proposes an efficient intrusion detection method, named Manta Ray Political Optimization (MRPO)-based Deep maxout network. Here, the training procedure of the Deep maxout network is achieved by the proposed MRPO algorithm that is derived by integrating Manta Ray Foraging Optimization (MRFO) and Political Optimizer (PO). Initially, different subsets of data are formed by partitioning the input data and each data subset is processed by slave nodes to pre-process the data. The wrapper-based model and fisher score are used to select features such that the fusion of these features is based on Hellinger distance. With selected features, the dimensionality of data is increased using the data augmentation and based on data augmented output, and Deep maxout network is applied to detect normal and abnormal behavior. The proposed method attains maximum testing accuracy, True Positive Rate (TPR), and True Negative Rate (TNR) of 0.939, 0.943, and 0.910 by the Apache webserver dataset. When the training data is 90, the accuracy of the proposed method is 9.15%, 6.38%, 5.75%, 2.66%, 2.12%, 1.49%, 074%, 1.70%, and 4.79% higher when compared to the existing approaches namely, Hybrid deep learning, RCCRO-FCM, HHO-based DBN, CGO+ensemble SVM, Deep maxout network, PO-based Deep maxout network, MRFO- based Deep maxout network, STL-HDL, and ExpSLO enabled DRN.

Minimum relevant features to obtain AI explainable system for predicting breast cancer in WDBC

The potential to explain why a machine learning model produces a certain prediction in incomprehensible terms is becoming increasingly crucial, as it provides accountability and confidence in the algorithm's decision-making process. The interpretation of complex models is difficult. Various approaches to dealing with this issue are being offered. These problems are typically handled in tree ensemble methods by assigning priority levels to input features globally or for a specific prediction. We show that current feature attribution approaches are inconclusive, and develop solutions using SHAP (SHapley Additive Explanation) values, LIME (Local Interpretable Model-Agnostic Explanations), and the Skope Rules package. We employ feature selection methods from SHAP and LIME in this work, which uses the Breast cancer Wisconsin data sets. In the suggested method, features are chosen at the first level of feature selection using Decision tree entropy values. Based on the SHAP and LIME reports, level 2 features are chosen from fewer options. The features are tested on a Decision Tree (DT) model and a DT and Support Vector Machine (SVM) ensemble. Experiments suggest that the ensemble works better as compared to DT. We have also used the Skope Rules package to generate global rules for generalization.

Novel ensemble of deep learning neural network and support vector machine for landslide susceptibility mapping in Tehri region, Garhwal Himalaya

Over the years, landslide has become one of the most destructive events that can happen in hilly areas. Tehri, a region in the Himalayas is no different. Current research aids in the construction of ensemble models of DLNN and SVM, which are then compared with various SVM kernels. Landslide susceptibility mapping in the Tehri region of the Himalayas has been worked upon using a deep learning (DLNN), four machine learnings (SVM-RBF, SVM-Linear, SVM-Polynomial, SVM-Sigmoid), and their novel ensembles i.e., DLNN with SVM-RBF, DLNN with SVM-Linear, DLNN with SVM-Polynomial and DLNN with SVM-Sigmoid. 16 geo-environmental landslide conditioning factors (LCFs) have been considered. These models were trained using 70% of inventory landslides and tested using 30% of the same. The results revealed the superiority of DLNN, DLNN-SVM (RBF), DLNN-SVM (Linear) models which quantified 28.32, 26.96 and 22.41% of the area highly susceptible for landslide, respectively.

An Ensemble of Gaussian Mixture Model and Support Vector Machines for Network Intrusion Detection

NetworkIntrusion Detection Systems (NIDS) can protect computer networks and computersystems by detecting abnormal network packets and taking agreed action plans,such as notifying an administrator or rejecting the network packets. In this study,the aim is the implementation of NIDS with improved performance using anensemble of Support Vector Machines (SVMs) and the Gaussian Mixture Model(GMM). Four SVMs with Radial Basis Function (RBF), linear, polynomial, andsigmoid kernel functions, and a GMM were trained with the same portion withKnowledge Discovery and Data Mining Tools Competition (KDD 99) dataset, andanother portion of the dataset was used to evaluate the performance of therespective NIDS models. Finally, the five models were integrated to form an ensembleIntrusion Detection System (IDS) model and the same test dataset was used tovalidate its performance. The IDS model of SVM with RBF kernel function has thebest performance with precision, recall, f1score, accuracy, false acceptance rate, and false rejection rate of 99.88,99.67, 99.77, 99.82, 0.08, and 0.33% respectively. The ensemble model built bycombining the five trained models where each of them has equal voting rightsyields state-of-art performance, precision, recall, f1-score, accuracy, falseacceptance rate, and false rejection rate of 99.7, 99.4, 99.55, 99.65, 0.18 and0.59% respectively though it is below the performance of the SVM-RBF and theSVM-polynomial models. Ensemble models are expected to have better performancethan a single classifier, but the result of this research shows that this isnot applicable in all cases as the SVM with RBF kernel outperformed theensemble classifier.

Ensemble of Support Vector Machine and Ontological Structures to Generate Abstractive Text Summarization

Automatic summarization systems are much needed to lessen the information overload which is being faced by people due to exponential growth of data on World Wide Web. These systems choose the most significant part of the text from a single document or multiple documents and present the compressed surrogate form of the complete information which was intended to be conveyed. In this research paper, we propose an approach to generate summary from a given text first by extracting the most relevant sentences and then making further concise by creating ontological structures of these sentences and then generating the abstractive summary from these structures. Our proposed system is evaluated with DUC 2002 data set and it is found that the performance of this system as evaluated using ROUGE-1 is 58.175 which is better than other state of the art systems. The values reported in the experimental process of the research report the significant contribution of this innovative method.

A data-centric weak supervised learning for highway traffic incident detection

Using the data from loop detector sensors for near-real-time detection of traffic incidents on highways is crucial to averting major traffic congestion. While recent supervised machine learning methods offer solutions to incident detection by leveraging human-labeled incident data, the false alarm rate is often too high to be used in practice. Specifically, the inconsistency in the human labeling of the incidents significantly affects the performance of supervised learning models. To that end, we focus on a data-centric approach to improve the accuracy and reduce the false alarm rate of traffic incident detection on highways. We develop a weak supervised learning workflow to generate high-quality training labels for the incident data without the ground truth labels, and we use those generated labels in the supervised learning setup for final detection. This approach comprises three stages. First, we introduce a data preprocessing and curation pipeline that processes traffic sensor data to generate high-quality training data through leveraging labeling functions, which can be domain knowledge-related or simple heuristic rules. Second, we evaluate the training data generated by weak supervision using three supervised learning models—random forest, k-nearest neighbors, and a support vector machine ensemble—and long short-term memory classifiers. The results show that the accuracy of all of the models improves significantly after using the training data generated by weak supervision. Third, we develop an online real-time incident detection approach that leverages the model ensemble and the uncertainty quantification while detecting incidents. Overall, we show that our proposed weak supervised learning workflow achieves a high incident detection rate (0.90) and low false alarm rate (0.08).

Brain Tumor Segmentation From Multimodal MRI Data Based on GLCM and SVM Classifier

The segmentation of MRI brain tumors utilizes computer technology to segment and label tumors and normal tissues automatically on multimodal brain images, which plays an important role in disease diagnosis, treatment planning, and surgical navigation. We propose a solution using gray-level co-occurrence matrix (GLCM) texture and an ensemble Support Vector Machine (SVM) structure.This manuscript per the authors focus on the effects of GLCM texture on brain tumor segmentation. The result is different from the application of the GLCM texture in other types of image processing.The experimental material was a dataset called BraTs2015. The segmented five different labels are normal brain, necrosis, edema, non-enhancing tumor, and enhancing tumor. The proposed model was verified with the Dice coefficient. The result demonstrated that this method has a better capacity and higher segmentation accuracy with a low computation cost.

A comparison of explainable artificial intelligence methods in the phase classification of multi-principal element alloys

We demonstrate the capabilities of two model-agnostic local post-hoc model interpretability methods, namely breakDown (BD) and shapley (SHAP), to explain the predictions of a black-box classification learning model that establishes a quantitative relationship between chemical composition and multi-principal element alloys (MPEA) phase formation. We trained an ensemble of support vector machines using a dataset with 1,821 instances, 12 features with low pair-wise correlation, and seven phase labels. Feature contributions to the model prediction are computed by BD and SHAP for each composition. The resulting BD and SHAP transformed data are then used as inputs to identify similar composition groups using k-means clustering. Explanation-of-clusters by features reveal that the results from SHAP agree more closely with the literature. Visualization of compositions within a cluster using Ceteris-Paribus (CP) profile plots show the functional dependencies between the feature values and predicted response. Despite the differences between BD and SHAP in variable attribution, only minor changes were observed in the CP profile plots. Explanation-of-clusters by examples show that the clusters that share a common phase label contain similar compositions, which clarifies the similar-looking CP profile trends. Two plausible reasons are identified to describe this observation: (1) In the limits of a dataset with independent and non-interacting features, BD and SHAP show promise in recognizing MPEA composition clusters with similar phase labels. (2) There is more than one explanation for the MPEA phase formation rules with respect to the set of features considered in this work.

A robust model selection framework for fault detection and system health monitoring with limited failure examples: Heterogeneous data fusion and formal sensitivity bounds

Fault detection models play a fundamental role in monitoring the health state of engineering systems subject to degradation processes. Data-driven fault detection models, albeit very effective when trained on large databases of failures, fail to perform well under a lack of failure examples. Because reliable engineering systems seldom fail, data shortage is often inevitable. To overcomes failure data scarcity, this work proposes a new model selection framework for the robust selection of fault detection and system health monitoring models. We combine heterogeneous sources of information (time-to-failure and sensors data), a model of the system structure, and mathematical bounds on the sensitivity and specificity of components fault classifiers. We use Support Vector Machines (SVMs) to detect components faults and Scenario theory to derive formal sensitivity and specificity bounds. The component predictions are combined within a system structure-function for system health states estimation. A novel model selection strategy optimizes the hyper-parameters of the SVM ensemble by minimizing system-level prediction errors and generalization error bounds of the individual fault classifiers. One of the main advantages of the proposed framework is a set of formal epistemic bounds on fault detection and false alarm probabilities quantifying the lack of data uncertainty affecting the fault detection rate. We test the method on three representative case studies: (1) On randomized fault detection experiments with synthetic data, (2) on ten SVM models for predictive maintenance of industrial health care imaging systems, and (3) on a real-world PHM challenge problem. The results prove the efficacy of the proposed approach and its usefulness for solving fault detection.