Support Vector Machine Ensemble Research Articles

Optimization of parallel SVM algorithm for big data

Parallel Support Vector Machine (SVM) based on big data has achieved some results in data mining, but due to the complexity of the data itself and a large amount of noisy data, its execution efficiency and classification accuracy in the big data environment are very low. In order to eliminate noise, a noise reduction method based on Noise Cleaning (NC) strategy was proposed, and redundant training samples in big data environments were deleted; Introduce an improved Artificial Fish Swarm Algorithm (IAFSA) to obtain the final Parallel SVM algorithm using mutual information and artificial fish swarm algorithm based on MapReduce (MIAFSA-PSVM) classification model. The results indicate that when compared to CMI-PSVM, the execution time of MIAFSA-PSVM algorithm is higher on the NDC dataset with the largest data size, The SVM parameter optimization algorithm based on MapReduce and cuckoo search (CSSVM-MR) and the particle swarm optimization based parallel support vector machine ensemble algorithm (PSO-PSVM) decreased by 40.1%, 79.3%, and 51.7%, respectively. This indicates that GIESVM-MR and MIAFSA-PSVM have strong adaptability to big data environments and high classification accuracy.

A novel approach for location selection in logistics: Ensemble SVM and improved genetic algorithm

SummaryGathering logistics operations in an individual location offers various benefits at the macro level namely reducing environmental and community issues, replacing overflow of traffic, minimizing air pollution, and more. But the complexity in the selection of location is maximized and the logistic operation chooses the wrong location due to frequent variations in characteristics. Also, it is suitable for capturing long‐term dependencies. To overcome these difficulties, we propose ESVM‐IGS; an Ensemble support vector machine, and an Improved Genetic algorithm, with an initial search strategy to improve the efficiency. The Ensemble SVM is applied to produce a better outcome. To identify optimal configurations in the complex optimization problem an initial search‐based improved genetic algorithm is implemented. To conduct our experiments, the ESVM‐IGS is rigorously evaluated on the GIS real‐time Dataset and the efficiency of the model is validated with various performance measures. From the analysis, the proposed method results that it solved the complexity burden and improved the selection ability of long‐term dependency. The experimental results depict the better efficiency of the ESVM‐IGS method for the location selection strategy of logistics.

An adaptive learning paradigm: event detection through a novel dynamic arithmetic optimization-based ensemble SVM for data stream classification

An adaptive learning paradigm: event detection through a novel dynamic arithmetic optimization-based ensemble SVM for data stream classification

Brain Damage Detection Using Ensemble Classifier for Children Health Progress

Magnetic Resonance Imaging (MRI) is a medical imaging technique used to detect brain damage, particularly in children. Previous classification techniques for brain damage had low accuracy, complex edge detection, and high noise levels. In this proposed work, use an ensemble classification technique to analyze MRI brain images of 6-10 year old children. The input data is preprocessed using a Median filter to remove noise and improve image quality before being sent to a segmentation technique. Each part of the segmented image, such as tissue, inner layer, overall shape, and texture, is classified based on pixel size. The classification is done using an ensemble of Support Vector Machines (SVM) and K-Nearest Neighbors (KNN) techniques. The ensemble prediction is multiplied with several input values to classify the MRI image, aggregating different layer values. The output classified image is validated and developed using Mat Lab software simulation, with an accuracy of 97.86%. This method can accurately classify various types of brain damage and aid in early detection in children.

Intrusion Detection System on Cloud Computing using Ensemble SVM

This study introduces a specialized Intrusion Detection System (IDS) designed specifically for cloud computing environments. By leveraging the UNSW_NB15 dataset, it employs feature selection techniques such as SelectKBest and ANOVA to extract relevant features, thereby improving the overall performance of the model. The IDS framework encompasses data preprocessing, Ensemble SVM model training, and performance evaluation utilizing standard metrics. The key methodology revolves around training SVM models using bagging and boosting techniques on preprocessed data, resulting in resilient intrusion detection models. These models are subsequently utilized to determine whether a given set of input features signifies a network intrusion. Through experimental analysis, the research demonstrates the system's efficacy in accurately detecting network intrusions, shedding light on its robustness and dependability.

Monitoring the after-effects of ischemic stroke through EEG microstates.

Stroke may cause extensive after-effects such as motor function impairments and disorder of consciousness (DoC). Detecting these after-effects of stroke and monitoring their changes are challenging jobs currently undertaken via traditional clinical examinations. These behavioural examinations often take a great deal of manpower and time, thus consuming significant resources. Computer-aided examinations of the electroencephalogram (EEG) microstates derived from bedside EEG monitoring may provide an alternative way to assist medical practitioners in a quick assessment of the after-effects of stroke. In this study, we designed a framework to extract microstate maps and calculate their statistical parameters to input to classifiers to identify DoC in ischemic stroke patients automatically. As the dataset is imbalanced with the minority of patients being DoC, an ensemble of support vector machines (EOSVM) is designed to solve the problem that classifiers always tend to be the majority classes in the classification on an imbalanced dataset. The experimental results show EOSVM get better performance (with accuracy and F1-Score both higher than 89%), improving sensitivity the most, from lower than 60% (SVM and AdaBoost) to higher than 80%. This highlighted the usefulness of the EOSVM-aided DoC detection based on microstates parameters. Therefore, the classifier EOSVM classification based on features of EEG microstates is helpful to medical practitioners in DoC detection with saved resources that would otherwise be consumed in traditional clinic checks.

Abstract 4907: RanBALL: Identifying B-cell acute lymphoblastic leukemia subtypes based on an ensemble random projection model

Abstract As the most common pediatric malignancy, B-cell acute lymphoblastic leukemia (B-ALL) has multiple distinct subtypes characterized by recurrent and sporadic somatic and germline genetic alterations like chromosomal alteration, transcription factor rearrangement or kinase inhibition. The treatment of B-ALL patients is personalized based on specific subtypes, as the treatment responses for different B-ALL subtypes may vary considerably. Identification of B-ALL subtypes can facilitate risk stratification and enable tailored therapeutic approaches. Existing methods for B-ALL subtyping primarily depend on immunophenotypic, cytogenetic and genomic analyses, which would be costly, complicated, and laborious in clinical practice applications. To overcome these challenges, we present RanBALL (an Ensemble Random Projection-Based Model for Identifying B-Cell Acute Lymphoblastic Leukemia Subtypes), an accurate and cost-effective model for B-ALL subtype identification based on transcriptomic profiling only. RanBALL leverages random projection (RP) to construct an ensemble of dimension-reduced multi-class classifiers for B-ALL subtyping. Specifically, the transcriptomic profiling features were projected onto low-dimensional spaces by random projection matrices whose elements conform to a distribution characterized by zero mean and unit variance. To ensure reliable and robust performance, we selected 20 subspace dimensions ranging from 600 to 2500, with intervals of 100. The transformed low dimensional data matrix was used for training an ensemble of multi-class support vector machine (SVM) classifiers, each corresponding to one of the RP matrices of various dimensions. The predicted probabilistic scores of each B-ALL subtype were integrated for determining the final decision. Results based on 10 times 10-fold cross validation tests for >1700 B-ALL patients demonstrated that the proposed model achieved an accuracy of 93.7%, indicating promising prediction capabilities of RanBALL for B-ALL subtyping. Furthermore, the 30% held-out tests suggested that the model was robust and consistent to maintain high confidence levels for accurate predictions. The high accuracies of RanBALL suggested that our model could effectively capture underlying patterns of transcriptomic profiling for accurate B-ALL subtype identification. To extend the impact of RanBALL, we have established a free and publicly available python package for RanBALL available at https://github.com/wan-mlab/RanBALL. We believe RanBALL will facilitate the discovery of B-ALL subtype-specific marker genes and therapeutic targets, and eventually have consequential positive impacts on downstream risk stratification and tailored treatment design. Citation Format: Lusheng Li, Hanyu Xiao, Joseph D. Khoury, Jieqiong Wang, Shibiao Wan. RanBALL: Identifying B-cell acute lymphoblastic leukemia subtypes based on an ensemble random projection model [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2024; Part 1 (Regular Abstracts); 2024 Apr 5-10; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2024;84(6_Suppl):Abstract nr 4907.

ENSEMBLE RESAMPLING SUPPORT VECTOR MACHINE, MULTINOMIAL REGRESSION TO MULTICLASS IMBALANCED DATA

Imbalanced data is a commonly encountered issue in classification analysis. This issue gives rise to prediction errors in the classification process, which in turn affects the sensitivity, particularly in the minority class. Resampling techniques can be employed as a means to mitigate the issue of Imbalanced data. Furthermore, ensemble approaches are Utilized in the classification procedure to augment the performance of classification. The present study assesses the efficacy of the bagging ensemble approach in conjunction with ADASYN as a means of addressing the aforementioned issue. The dataset Utilized in this work comprises Imbalanced Glass Identification data, Imbalanced Iris data, and Imbalanced synthetic data. The study Centres on the Utilization of Support Vector Machines (SVM) with parameter optimization using repeated cross-validation (k = 10) and the application of multinomial regression. The evaluation of classification outcomes involves a comparison between the ensemble technique and multinomial regression. This comparison is conducted under pre- and post-resampling conditions, with the evaluation metrics being accuracy, sensitivity, and specificity. The analysis of classification outcomes across the three datasets suggests that the ensemble resampling SVM approach and multinomial regression exhibit superior performance compared to the ensemble SVM and multinomial regression approaches when applied to non-resampled data. Resampling of data has been observed to enhance sensitivity, particularly in the minority class.

A MENTAL ILLNESS EARLY WARNING ANALYSIS MODEL USING AN IMPROVED SVM ALGORITHM AND EEG SIGNALS

The fast-paced lifestyle, intense competition, and constantly changing environment of modern society are important factors contributing to the emergence of psychological disorders. Factors such as work pressure, family conflicts, and interpersonal relationship difficulties pose challenges to individual mental health. Prolonged use of electronic devices and the widespread use of social media further exacerbate mental health issues, as the constant influx of information makes it difficult for people to relax and focus. The harm of psychological disorders to individuals and society cannot be ignored. It not only diminishes the quality of life for patients but also affects work efficiency and social participation. In extreme cases, psychological disorders can lead to serious consequences like suicide, causing immense pain to families and society. Therefore, prioritizing mental health, raising awareness of mental well-being, and timely intervention and treatment of psychological disorders are crucial for individual happiness and social stability. Based on this, this study employs an improved Support Vector Machine (SVM) and Electroencephalogram (EEG) signals for the diagnosis of psychological disorders, aiming for more effective treatment of mental health conditions. Specifically, the EEG signals of the experimental subjects are first collected as training data for the model. The dataset undergoes preprocessing operations such as cleaning and normalization. Due to the limited availability of published information about individuals with psychological disorders, an ensemble SVM algorithm is used as a warning model to improve the accuracy of psychological disorder diagnosis. This algorithm utilizes a differential measurement method to adjust the selection of SVM parameters. The training sample size remains constant, with several iterations of training performed. By combining the differential measurement method, a specific parameter combination is obtained, maximizing the differentiation between base classifiers. This ultimately addresses issues such as small sample size and insufficient sample information in the dataset. Experimental results demonstrate that the model used in this study exhibits good performance in diagnosing psychological disorders.

Classification of human walking context using a single-point accelerometer

Real-world walking data offers rich insights into a person’s mobility. Yet, daily life variations can alter these patterns, making the data challenging to interpret. As such, it is essential to integrate context for the extraction of meaningful information from real-world movement data. In this work, we leveraged the relationship between the characteristics of a walking bout and context to build a classification algorithm to distinguish between indoor and outdoor walks. We used data from 20 participants wearing an accelerometer on the thigh over a week. Their walking bouts were isolated and labeled using GPS and self-reporting data. We trained and validated two machine learning models, random forest and ensemble Support Vector Machine, using a leave-one-participant-out validation scheme on 15 subjects. The 5 remaining subjects were used as a testing set to choose a final model. The chosen model achieved an accuracy of 0.941, an F1-score of 0.963, and an AUROC of 0.931. This validated model was then used to label the walks from a different dataset with 15 participants wearing the same accelerometer. Finally, we characterized the differences between indoor and outdoor walks using the ensemble of the data. We found that participants walked significantly faster, longer, and more continuously when walking outdoors compared to indoors. These results demonstrate how movement data alone can be used to obtain accurate information on important contextual factors. These factors can then be leveraged to enhance our understanding and interpretation of real-world movement data, providing deeper insights into a person’s health.

Maximizing Emotion Recognition Accuracy with Ensemble Techniques on EEG Signals

Background: Emotion is a strong feeling such as love, anger, fear, etc. Emotion can be recognized in two ways, i.e., External expression and Biomedical data-based. Nowadays, various research is occurring on emotion classification with biomedical data. Aim: One of the most current studies in the medical sector, gaming-based applications, education sector, and many other domains is EEG-based emotion identification. The existing research on emotion recognition was published using models like KNN, RF Ensemble, SVM, CNN, and LSTM on biomedical EEG data. In general, only a few works have been published on ensemble or concatenation models for emotion recognition on EEG data and achieved better results than individual ones or a few machine learning approaches. Various papers have observed that CNN works better than other approaches for extracting features from the dataset, and LSTM works better on the sequence data. Method: Our research is based on emotion recognition using EEG data, a mixed-model deep learning methodology, and its comparison with a machine learning mixed-model methodology. In this study, we introduced a mixed model using CNN and LSTM that classifies emotions in valence and arousal on the DEAP dataset with 14 channels across 32 people. Result and Discussion: We then compared it to SVM, KNN, and RF Ensemble, and concatenated these models with it. First preprocessed the raw data, then checked emotion classification using SVM, KNN, RF Ensemble, CNN, and LSTM individually. After that with the mixed model of CNN-LSTM, and SVM-KNN-RF Ensemble results are compared. Proposed model results have better accuracy as 80.70% in valence than individual ones with CNN, LSTM, SVM, KNN, RF Ensemble and concatenated models of SVM, KNN and RF Ensemble. Conclusion: Overall, this paper concludes a powerful technique for processing a range of EEG data is the combination of CNNs and LSTMs. Ensemble approach results show better performance in the case of valence at 80.70% and 78.24% for arousal compared to previous research.

Ensemble of Deep Features for Breast Cancer Histopathological Image Classification

Abstract Analysis of histopathological images (HIs) is crucial for detecting breast cancer (BR). However, because they vary, it is still very difficult to extract well-designed elements. Deep learning (DL) is a recent development that is used to extract high-level features. However, DL techniques continue to confront several difficult problems, such as the need for sufficient training data for DL models, which reduces the classification findings. In this study, an ensemble deep transfer convolutional neural network is presented to address this problem. The pre-trained models (ResNet50 and MobileNet) are employed to extract high-level features by freezing the front layer parameters while fine-tuning the last layers. In the proposed ensemble framework, KNN, SVM, logistic regression and neural networks are used as base classifiers. The majority vote and product approaches are used to integrate the predictions of each separate classifier. In the benchmark BreaKHis dataset, the suggested ensemble model is compared to some current approaches. It demonstrates that while the ensemble model obtains a considerable accuracy of 97.72% for the multiclass classification test, it achieves an accuracy of 99.2% for the binary task. The suggested ensemble model’s effectiveness in extracting useful features for BR images is demonstrated by comparison with existing cutting-edge models.

A Non-destructive Method for the determination of Carbonation Time for Nominal Concrete Cover Depth Using Non-Linear En-semble Prediction

Carbonation, a process involving the reaction of carbon dioxide and moisture, results in the for-mation of powdery calcium carbonate, a critical durability issue causing reinforcement corrosion. The study analyzed carbonation data from coastal and inland buildings in the Turkish Republic of Northern Cyprus, re-vealing higher carbonation rates than anticipated within their lifespan. An artificial intelligence model named Support Vector Machine (SVM) was applied to predict carbonation time (T) to penetrate concrete cover of 25mm in the TRNC. Subsequently used two ensemble techniques, namely Neural Network Ensembles (NNE) and Support Vector Machine Ensembles (SVME) to enhance the performance of the prediction of T. Four performance criteria namely Correlation Coefficient (CC), Root Mean Square Error (RMSE), Correlation Co-efficient (R2), Mean Absolute Error (MAE) was applied to verify the modelling accuracy. The Values of R2 of Ensemble techniques indicated significant increase in the performance, greater than the SVM model. This shows that using ensemble techniques is promising in getting better predictions of carbonation time (T) to penetrate concrete cover. The results obtained showed that NNE and SVME combination demonstrated the best performance under the evaluation criteria of R2 = 0.8721 and R2 = 0.8644 in testing phases respectively in comparison SVM-M1 to SVM-M3.

Vibration-Based Fault Diagnosis for Railway Point Machines Using Multi-Domain Features, Ensemble Feature Selection and SVM

Vibration-Based Fault Diagnosis for Railway Point Machines Using Multi-Domain Features, Ensemble Feature Selection and SVM

Bounded exponential loss function based AdaBoost ensemble of OCSVMs

As a commonly used ensemble method, AdaBoost has drawn much consideration in the field of machine learning. However, AdaBoost is highly sensitive to outliers. The performance of AdaBoost may be greatly deteriorated when the training samples are polluted by outliers. For binary and multi-class classifications, there have emerged many approaches to improving the robustness of AdaBoost against outliers. Unfortunately, there are too few researches on enhancing the robustness of AdaBoost against outliers in the case of one-class classification. In this study, the exponential loss function of AdaBoost is replaced by a more robust one to improve the anti-outlier ability of the conventional AdaBoost based ensemble of one-class support vector machines (OCSVMs). Furthermore, based on the redesigned loss function, the update formulae for the weights of base classifiers and the probability distribution of training samples are reformulated towards the AdaBoost ensemble of OCSVMs. The empirical error upper bound is derived from the theoretical viewpoint. Experimental outcomes upon the artificial and benchmark data sets show that the presented ensemble approach is more robust against outliers than its related methods.

An improved ensemble machine learning classifier for efficient spectrum sensing in cognitive radio networks

SummaryCognitive radio network (CRN) is one form of wireless communication for solving the spectrum underutilization problem. It is mainly used for sensing and learning the electromagnetic environment and changes according to the environment. CRNs mainly depend on the cooperation functionality for making the network work efficiently. The main technology in cognitive radio is spectrum sensing. The cooperative spectrum sensing (CSS) scheme is used for estimating the high transmit power in the cognitive radio networks, and the result is that it satisfies the interference constraints. It develops the communication overhead for the local observation dissemination between the secondary users. It is used for speed and accurate detection techniques of the user, and it also identifies the spectrum holes without any interference to others while sharing with other users. It is based on the ensemble support vector machine (SVM) with CSS for producing high performance in CRNs with the use of artificial intelligence (AI) techniques.

Hybrid ABC and black hole algorithm with genetic operators optimized SVM ensemble based diagnosis of breast cancer

Hybrid ABC and black hole algorithm with genetic operators optimized SVM ensemble based diagnosis of breast cancer

Identification and Automated Delineation of Radioresistant Biological Tumor Volume in Nasopharyngeal Carcinoma Based on Magnetic Resonance Imaging Radiomics

Widespread use of intensity modulated radiotherapy (IMRT) has improved the tumor control rate of nasopharyngeal carcinoma (NPC). However, nearly 20% of the patients with local-advanced NPC would relapse after precise irradiation and 80% of the recurrent lesions occur within the high dose field, suggesting that there are radiation-resistant cancer cell subsets within the tumor. In this context, identification and contouring of radiation resistance region of NPC for dose escalation at primary IMRT could be advantageous. In this work, we proposed a two-step radiomics workflow to predict local relapse and the recurrent region of NPC before primary IMRT. In this single-center, retrospective study, pre-treatment magnetic resonance (MR) sequences of T1-weighted imaging (T1-w) and contrast-enhanced T1-weighted imaging (CET1-w) were collected from 800 patients of newly diagnosed and non-metastatic NPC between April 2009 and December 2015. The primary gross tumor volume (GTVp) of all patients and the actual recurrent lesion (GTVr) of patients who suffered from local recurrence were manually contoured for further analysis. A two-step complete radiomics workflow was designed to predict tumor recurrence and segment the region. First, least absolute shrinkage and selection operator (LASSO) was utilized for radiomics features selection of GTVp and support vector machine (SVM) was adopted to predict the recurrence. If the model predicts a recurrence, then the workflow utilizes an improved 3D U-Net to segment the recurrent region. Area under receiver operating characteristic curve (ROC-AUC) was used to evaluate the performance of tumor recurrence prediction, and Dice similarity coefficient (DSC) was used to assess the consistence between the actual and predicted GTVr. Of 800 NPC patients, 95 (11.9%) patients developed in-field local recurrence. For recurrence risk prediction, the SVM ensemble model (T1-w+CET1-w) was selected for further application with higher sensitivity. The average ROC-AUC, specificity, sensitivity of the SVM ensemble model in a 5-fold cross-validation and in the independent test set of 160 patients were 0.922, 0.922, 0.777 and 0.928, 0.915, 0.737, respectively. Moreover, for recurrent region segmentation, the multi-modality (T1-w+CET1-w) model was superior to the single-modality (T1-w or CET1-w) model. In an independent test set of 15 patients, the DSC, sensitivity and 95% Hausdorff Distance between actual and predicted GTVr was 0.549±0.176, 0.696±0.118 and 9.813±4.788 which was superior to 0.444±0.188, 0.497±0.218 and 12.047±5.361 of original 3D U-Net. The proposed two-step radiomics workflow showed a good performance in predicting tumor recurrence of NPC. The predicted location of the recurrence lesion was all accurate, but there was still a certain difference between the volume of the automated delineated and actual GTVr, which needed to be further optimized to be used as biological tumor volume.

Closed loop BCI system for Cybathlon 2020

ABSTRACT We developed a Brain-Computer Interface (BCI) System for the BCI discipline of Cybathlon 2020 competition, where participants with tetraplegia (pilots) control a computer game with mental commands. To extract features from one-second-long electroencephalographic (EEG) signals, we calculated the absolute of the Fast-Fourier Transformation amplitude (FFTabs) and introduced two methods: Feature Average and Feature Range. The former calculates the average of the FFTabs for a specific frequency band, while the later generates multiple Feature Averages for non-overlapping 2 Hz wide frequency bins. The resulting features were fed to a Support Vector Machine classifier and tested on the PhysioNet database and our dataset containing 16 offline experiments recorded with the help of 2 pilots. 27 gameplay trials (out of 59) with our pilots reached the 240-second qualification time limit, which demonstrates the usability of our system in real-time circumstances. We critically compared the Feature Average of canonical frequency bands (alpha, beta, gamma, and theta) with our suggested range30 and range40 methods. On the PhysioNet dataset, the range40 method combined with an ensemble SVM classifier significantly reached the highest accuracy level (0.4607), with a 4-class classification; moreover, it outperformed the state-of-the-art EEGNet.

Classification of Acute Lymphocytic Leukemic Blood Cell Images using Hybrid CNN-Enhanced Ensemble SVM Models and Machine Learning Classifiers

Acute Lymphocytic Leukemia is a dangerous kind of malignant cancer caused due to the overproduction of white blood cells. The white blood cells in our body are responsible for fighting against infections, if the WBC increases the immunity will decrease and it would lead to serious health conditions. Malignant cancers such as ALL is life threatening if the disease is not diagnosed at an early stage. If a person is suffering from ALL the disease needs to be diagnosed at an early stage before it starts spreading, if it starts spreading the person’s chances of survival would also reduce. Here comes the need of an accurate automated system which would assist the oncologists to diagnose the disease as early as possible. In this paper some of the algorithms that are enhanced to detect and classify ALL are incorporated. In order to classify the Acute Lymphocytic Leukemia a hybrid model has been deployed to improve the accuracy of the diagnosis and it is termed as Hybrid CNN Enhanced Ensemble SVM for the classification of malignancy. Machine Learning classifiers are also used to design the system and it is then compared with enhanced CNN based on the performance metrics.