Support Vector Machine Ensemble Model Research Articles

Early warning model based on support vector machine ensemble algorithm

Support Vector Machine (SVM) is a powerful machine learning technique often applied in various early warning scenarios. When using SVM for binary classification, its output results can be converted into a probability distribution based on sample categories. Through the analysis of the results, it is found that when the probability of a sample belonging to a positive class is equivalent to the probability of a sample belonging to a negative class, there is a high possibility of misclassification of the sample. To address this issue, this study proposes a new SVM ensemble model construction method. We generate SVM sub-classifiers to extract error prone set, and utilise three different combination rules to construct ensemble model based on the output probability of the samples. The accuracy of the whole data set is improved for the classification accuracy of the easily misclassified data are improved. This research evaluated the model based on Wind turbine SCADA data and UCI benchmark datasets. According to the research results, the accuracy rate of SCADA data is as high as 97.52%, and the accuracy rate of German credit data is 77.50%. Meanwhile, the comparison with other methods also validates the effectiveness of the proposed method.

Identification and Automated Delineation of Radioresistant Biological Tumor Volume in Nasopharyngeal Carcinoma Based on Magnetic Resonance Imaging Radiomics

Widespread use of intensity modulated radiotherapy (IMRT) has improved the tumor control rate of nasopharyngeal carcinoma (NPC). However, nearly 20% of the patients with local-advanced NPC would relapse after precise irradiation and 80% of the recurrent lesions occur within the high dose field, suggesting that there are radiation-resistant cancer cell subsets within the tumor. In this context, identification and contouring of radiation resistance region of NPC for dose escalation at primary IMRT could be advantageous. In this work, we proposed a two-step radiomics workflow to predict local relapse and the recurrent region of NPC before primary IMRT. In this single-center, retrospective study, pre-treatment magnetic resonance (MR) sequences of T1-weighted imaging (T1-w) and contrast-enhanced T1-weighted imaging (CET1-w) were collected from 800 patients of newly diagnosed and non-metastatic NPC between April 2009 and December 2015. The primary gross tumor volume (GTVp) of all patients and the actual recurrent lesion (GTVr) of patients who suffered from local recurrence were manually contoured for further analysis. A two-step complete radiomics workflow was designed to predict tumor recurrence and segment the region. First, least absolute shrinkage and selection operator (LASSO) was utilized for radiomics features selection of GTVp and support vector machine (SVM) was adopted to predict the recurrence. If the model predicts a recurrence, then the workflow utilizes an improved 3D U-Net to segment the recurrent region. Area under receiver operating characteristic curve (ROC-AUC) was used to evaluate the performance of tumor recurrence prediction, and Dice similarity coefficient (DSC) was used to assess the consistence between the actual and predicted GTVr. Of 800 NPC patients, 95 (11.9%) patients developed in-field local recurrence. For recurrence risk prediction, the SVM ensemble model (T1-w+CET1-w) was selected for further application with higher sensitivity. The average ROC-AUC, specificity, sensitivity of the SVM ensemble model in a 5-fold cross-validation and in the independent test set of 160 patients were 0.922, 0.922, 0.777 and 0.928, 0.915, 0.737, respectively. Moreover, for recurrent region segmentation, the multi-modality (T1-w+CET1-w) model was superior to the single-modality (T1-w or CET1-w) model. In an independent test set of 15 patients, the DSC, sensitivity and 95% Hausdorff Distance between actual and predicted GTVr was 0.549±0.176, 0.696±0.118 and 9.813±4.788 which was superior to 0.444±0.188, 0.497±0.218 and 12.047±5.361 of original 3D U-Net. The proposed two-step radiomics workflow showed a good performance in predicting tumor recurrence of NPC. The predicted location of the recurrence lesion was all accurate, but there was still a certain difference between the volume of the automated delineated and actual GTVr, which needed to be further optimized to be used as biological tumor volume.

Two-Stage PNN–SVM Ensemble for Higher Education Admission Prediction

In this paper, we investigate the methods used to evaluate the admission chances of higher education institutions’ (HEI) entrants as a crucial factor that directly influences the admission efficiency, quality of education results, and future students’ life-long trajectories. Due to the conditions of uncertainty surrounding the decision-making process that determines the admission of entrants and the inability to independently assess the probability of potential outcomes, we propose the application of the machine learning (ML) model as an algorithm that provides decision-making support. The proposed model includes the support vector machine (SVM) stacking ensemble, which expands the input data set obtained using the Probabilistic Neural Network (PNN). The basic algorithms include four SVM ensemble methods with different kernel functions and Logistic Regression (LR) as a meta-algorithm. We evaluate the accuracy of the developed model in three stages: comparison with existing ML methods; comparison with a single-based model that comprises it; and comparison with a similar stacking model and with other types of ensembles (boosting, begging). The results of the designed two-stage PNN–SVM ensemble model provided an accuracy of 94% and possessed acquired superiority in the comparison stages. The obtained results enable the use of the presented model in the subsequent stages of the development of an intellectual support system for decision making regarding entrants’ admission.

Annual forecasting of high‐temperature days in China through grey wolf optimization‐based support vector machine ensemble

AbstractWith the intensification of anthropogenic warming and urbanization, high‐temperature weather poses an enormous threat to socio‐economic and human healthy. However, the studies on annual high‐temperature days forecasting based on machine learning are relatively deficient. This study proposes a support vector machine (SVM) ensemble model based on grey wolf optimization (GWO) to predict annual high‐temperature days in Guangzhou, Shanghai and Beijing of China. Atmospheric circulation indices during 1959–2013 were utilized as inputs to train and validate models. The fivefold cross validation was used to expand the sample data and evaluate the performance of the member and ensemble models. The optimal ensemble model for Guangzhou has the highest average R (0.8939) and the lowest average root mean square error (RMSE; 3.3771), followed by the optimal ensemble models for Beijing (0.8871 and 3.6059) and Shanghai (0.7578 and 3.9968). Furthermore, compared with the typical SVM and optimal member models, the average validation RMSE of the optimal ensemble model was improved by 32.6 and 10.0% for Guangzhou, by 29.8 and 9.1% for Shanghai, and by 41.3 and 15.1% for Beijing, respectively. This study demonstrates that the GWO‐based SVM ensemble model can be a promising tool for annual high‐temperature days forecasting due to the nonlinear fitting power of the SVM, the hyperparameters tuning capability of the GWO algorithm, and the integration ability of ensemble learning.

Dynamic correction of soft measurement model for evaporation process parameters based on ARMA.

To address the significant soft measurement errors in traditional static models for evaporation process parameters, which are characterized by continuity and cumulativity, this paper proposes a dynamic correction method for soft measurement models of evaporation process parameters based on the autoregressive moving-average model (ARMA). Initially, the Powell's directional evolution (Powell-DE) algorithm is utilized to identify the autoregressive order and moving average order of the ARMA model. Subsequently, the prediction error of a mechanism-reduced robust least squares support vector machine ensemble model is utilized as input. An error time series prediction model, which compensates for the errors in the autoregressive moving average model, is then applied for dynamic estimation of the prediction error. Finally, an integration strategy using the entropy method is employed to combine the static soft measurement model, based on the mechanism-reduced robust least squares support vector machine, with the dynamic correction soft measurement model, which is based on the error time series compensation of the ARMA model. The new model is analyzed and validated using production data from an alumina plant's evaporation process. Compared to traditional models, the new model demonstrates significantly improved prediction accuracy and is capable of dynamic prediction of evaporation process parameters.

Mapping dead understoreyBuxus hyrcana Pojarkusing Sentinel-2 and Sentinel-1 data

AbstractThe Hyrcanian Forests comprise a continuous 800-km belt of mostly deciduous broadleaf forests and are considered as Iran’s most important vegetation region in terms of density, canopy cover and species diversity. One of the few evergreen species of the Hyrcanian Forests is the box tree (Buxus), which is seriously threatened by box blight disease and box tree moth outbreaks. Therefore, information on the spatial distribution of intact and infested box trees is essential for recovery monitoring, control treatment and management. To address this critical knowledge gap, we integrated a genetic algorithm (GA) with a support vector machine (SVM) ensemble classification based on the combination of leaf-off optical Sentinel-2 and radar Sentinel-1 data to map the spatial distribution of box tree mortality. We additionally considered the overstorey species composition to account for a potential impact of overstory stand composition on the spectral signature of understorey defoliation. We consequently defined target classes based on the combination of dominant overstorey trees (using two measures including the relative frequency and the diameter at breast height) and two defoliation levels of box trees (including dead and healthy box trees). Our classification workflow applied a GA to simultaneously derive optimal vegetation indices (VIs) and tuning parameters of the SVM. Then the distribution of box tree defoliation was mapped by an SVM ensemble with bagging using GA-optimized VIs and radar data. The GA results revealed that normalized difference vegetation index, red edge normalized difference vegetation index and green normalized difference vegetation index were appropriate for box tree defoliation mapping. An additional comparison of GA-SVM (using GA-optimized VIs and tuning parameters) with a simple SVM (using all VIs and user-based tuning parameters) showed that our suggested workflow performs notably better than the simple SVM (overall accuracy of 0.79 vs 0.74). Incorporating Sentinel-1 data to GA-SVM, marginally improved the performance of the model (overall accuracy: 0.80). The SVM ensemble model using Sentinel-2 and -1 data yielded high accuracies and low uncertainties in mapping of box tree defoliation. The results showed that infested box trees were mostly located at low elevations, low slope and facing north. We conclude that mortality of evergreen understorey tree species can be mapped with good accuracies using freely available satellite data if a suitable work-flow is applied.

A band selection approach based on wavelet support vector machine ensemble model and membrane whale optimization algorithm for hyperspectral image

Hyperspectral Image (HSI) has become one of the important remote sensing sources for object interpretation by its abundant band information. Among them, band selection is considered as the main theme in HSI classification to reduce the data dimension, and it is a combinatorial optimization problem and difficult to be completely solved by previous techniques. Whale Optimization Algorithm (WOA) is a newly proposed swarm intelligence algorithm that imitates the predatory strategy of humpback whales, and membrane computing is able to decompose the band information into a series of elementary membranes that decreases the coding length. In addition, Support Vector Machine (SVM) combined with wavelet kernel is adapted to HSI datasets with high dimension and small samples, ensemble learning is an effective tool that synthesizes multiple sub-classifiers to solve the same problem and obtains accurate category label for each sample. In the paper, a band selection approach based on wavelet SVM (WSVM) ensemble model and membrane WOA (MWOA) is proposed, experimental results indicate that the proposed HSI classification technique is superior to other corresponding and newly proposed methods, achieves the optimal band subset with a fast convergence speed, and the overall classification accuracy has reached 93% for HSIs.

Machine Learning Paradigm towards Content Based Image Retrieval on High Resolution Satellite Images

In the current era, content based image retrieval based on pattern recognition and classification using machine learning paradigm is an innovative way. In order to retrieve high resolution satellite images Support Vector Machine (SVM) a machine learning paradigm is helpful for learning process and for pattern recognition and classification; ensemble methods give better machine learning results. In this paper, SVM based on random subspace and boosting ensemble learning is proposed for very high resolution satellite image retrieval. The learned SVM ensemble model is used to identify the images that most similar informative for active learning. A bias-weighting system is developed to direct the ensemble model to pay more attention on the positive examples than the negative ones. The UCMerced land use satellite image dataset is used for experimental work. Accuracy and error rate are found to be precise. The tentative effects illustrate that the proposed model derived enhanced retrieval accurateness at the optimum level as well as significantly more effective than existing approaches. The proposed method can diminish the gap dimensionality and conquer the difficulty. The comparisons are evaluated by using precision and recall measurements. Comparative analysis observed that the retrieval time for a particular image have been reduced and the precision is increased. The primary aim of this paper is to represent the significance of ensemble learning with support vector machine in efficient retrieval of image.

A novel optimal support vector machine ensemble model for NOX emissions prediction of a diesel engine

A novel ensemble method based on principal component analysis (PCA), genetic algorithm (GA) and support vector machine (SVM) implemented in MATLAB® is presented for establishing the NOX emissions prediction model for a diesel engine for both steady and transient operating states. The different stages of data preprocessing, modeling, optimization and prediction were discussed in detail. Normalization and PCA were used to reduce differences and redundancy of the datasets respectively. Subsequently, the SVM model was trained with 1/3 of the equi-spaced data samples (a simple DoE) selected after preprocessing. A grid search and GA were then applied as the combination strategy with the fitness function being the cross-validated root mean square error (RMSE) for optimizing the model parameters to improve the prediction accuracy. The optimal model was finally tested using the rest 2/3 data samples. Compared with other three methods, the proposed model exhibited superior accuracy both on training and testing datasets.

A novel least squares support vector machine ensemble model for NOx emission prediction of a coal-fired boiler

Real operation data of power plants are inclined to be concentrated in some local areas because of the operators’ habits and control system design. In this paper, a novel least squares support vector machine (LSSVM)-based ensemble learning paradigm is proposed to predict NOx emission of a coal-fired boiler using real operation data. In view of the plant data characteristics, a soft fuzzy c-means cluster algorithm is proposed to decompose the original data and guarantee the diversity of individual learners. Subsequently the base LSSVM is trained in each individual subset to solve the subtask. Finally, partial least squares (PLS) is applied as the combination strategy to eliminate the collinear and redundant information of the base learners. Considering that the fuzzy membership also has an effect on the ensemble output, the membership degree is added as one of the variables of the combiner. The single LSSVM and other ensemble models using different decomposition and combination strategies are also established to make a comparison. The result shows that the new soft FCM-LSSVM-PLS ensemble method can predict NOx emission accurately. Besides, because of the divide and conquer frame, the total time consumed in the searching the parameters and training also decreases evidently.

신용카드 대손회원 예측을 위한 SVM 모형

In this paper, credit card delinquency means the possibility of occurring bad debt within the certain near future from the normal accounts that have no debt and the problem is to predict, on the monthly basis, the occurrence of delinquency 3 months in advance. This prediction is typical binary classification problem but suffers from the issue of data imbalance that means the instances of target class is very few. For the effective prediction of bad debt occurrence, Support Vector Machine (SVM) with kernel trick is adopted using credit card usage and payment patterns as its inputs. SVM is widely accepted in the data mining society because of its prediction accuracy and no fear of overfitting. However, it is known that SVM has the limitation in its ability to processing the large-scale data. To resolve the difficulties in applying SVM to bad debt occurrence prediction, two stage clustering is suggested as an effective data reduction method and ensembles of SVM models are also adopted to mitigate the difficulty due to data imbalance intrinsic to the target problem of this paper. In the experiments with the real world data from one of the major domestic credit card companies, the suggested approach reveals the superior prediction accuracy to the traditional data mining approaches that use neural networks, decision trees or logistics regressions. SVM ensemble model learned from T2 training set shows the best prediction results among the alternatives considered and it is noteworthy that the performance of neural networks with T2 is better than that of SVM with T1. These results prove that the suggested approach is very effective for both SVM training and the classification problem of data imbalance.

A Hybrid Support Vector Machine Ensemble Model for Credit Scoring

Credit risk is the most challenging risk to which financial institution are exposed. Credit scoring is the main analytical technique for credit risk assessment. In this paper a hybrid model for credit scoring is designed which applies ensemble learning for credit granting decisions. The hybrid model combines clustering and classification techniques. Ten Support Vector Machine (SVM) classifiers are utilized as the members of ensemble model. Since even a small improvement in credit scoring accuracy causes significant loss reduction, then the application of ensemble in hybrid model leads to better performance of classification. A real dataset is used to test the model performance. The test results shows that proposed hybrid SVM ensemble has better classification accuracy when compared with other methods.

Support vector machine ensemble model based on KFCM and its application

To further enhance the regression prediction accuracy of support vector machine,a Least Squares Support Vector Machine(LS-SVM) ensemble model based on Kernel Fuzzy C-Means clustering(KFCM) was proposed.The KFCM algorithm was used to classify LS-SVMs trained independently by its output on validate samples,the generalization errors of LS-SVMs in each category to the validate set were calculated of the LS-SVM whose error was minimum would be selected as the representative of its category,and then the final prediction was obtained by simple average of the predictions of the component LS-SVM.The experiments in short-term load forecasting show the proposed approach has higher accuracy.