Support Vector Machine Ensemble Research Articles

Time-frequency analysis of speech signals using the Stockwell transform for the detection of upper respiratory tract infection

The acoustic properties of speech demonstrate modifications in the presence of different health states. Biomedical engineering has great promise for creating non-invasive diagnostic processes that use speech as a biomarker. The use of speech indications to screen for upper respiratory tract infections (URTIs), such as the common cold, may have potential advantages in terms of limiting transmission. In this study, we have employed the Stockwell transform -based time-frequency (TF) analysis of speech signals for URTI detection. The Stockwell transform is applied on speech signals to derive their TF representation. Using a TF matrix, the various statistics of magnitude and phase are calculated and used as features for classifying speech of healthy speakers and speakers with URTI. The URTIC database is employed for evaluating the proposed features. The utilization of an ensemble of support vector machines (SVM) is proposed as a classification approach to address the issue of class imbalance. The results show that the proposed method produces comparable outcomes to state-of-the-art approaches. The proposed features obtain 66.53% and 64.65% UARs on the development and test partitions of the URTIC database.

RanBALL: An Ensemble Random Projection Model for Identifying Subtypes of B-cell Acute Lymphoblastic Leukemia.

As the most common pediatric malignancy, B-cell acute lymphoblastic leukemia (B-ALL) has multiple distinct subtypes characterized by recurrent and sporadic somatic and germline genetic alterations. Identification of B-ALL subtypes can facilitate risk stratification and enable tailored therapeutic approaches. Existing methods for B-ALL subtyping primarily depend on immunophenotypic, cytogenetic and genomic analyses, which would be costly, complicated, and laborious in clinical practice applications. To overcome these challenges, we present RanBALL (an Ensemble Ran dom Projection-Based Model for Identifying B -Cell A cute L ymphoblastic L eukemia Subtypes), an accurate and cost-effective model for B-ALL subtype identification based on transcriptomic profiling only. RanBALL leverages random projection (RP) to construct an ensemble of dimension-reduced multi-class support vector machine (SVM) classifiers for B-ALL subtyping. Results based on 100 times 5-fold cross validation tests for >1700 B-ALL patients demonstrated that the proposed model achieved an accuracy of 93.35%, indicating promising prediction capabilities of RanBALL for B-ALL subtyping. The high accuracies of RanBALL suggested that our model could effectively capture underlying patterns of transcriptomic profiling for accurate B-ALL subtype identification. We believe RanBALL will facilitate the discovery of B-ALL subtype-specific marker genes and therapeutic targets, and eventually have consequential positive impacts on downstream risk stratification and tailored treatment design.

The role of artificial intelligence in disease prediction: using ensemble model to predict disease mellitus.

The traditional complications of diabetes are well known and continue to pose a considerable burden to millions of people with diabetes mellitus (DM). With the continuous accumulation of medical data and technological advances, artificial intelligence has shown great potential and advantages in the prediction, diagnosis, and treatment of DM. When DM is diagnosed, some subjective factors and diagnostic methods of doctors will have an impact on the diagnostic results, so the use of artificial intelligence for fast and effective early prediction of DM patients can provide decision-making support to doctors and give more accurate treatment services to patients in time, which is of great clinical medical significance and practical significance. In this paper, an adaptive Stacking ensemble model is proposed based on the theory of "error-ambiguity decomposition," which can adaptively select the base classifiers from the pre-selected models. The adaptive Stacking ensemble model proposed in this paper is compared with KNN, SVM, RF, LR, DT, GBDT, XGBoost, LightGBM, CatBoost, MLP and traditional Stacking ensemble models. The results showed that the adaptive Stacking ensemble model achieved the best performance in five evaluation metrics: accuracy, precision, recall, F1 value and AUC value, which were 0.7559, 0.7286, 0.8132, 0.7686 and 0.8436. The model can effectively predict DM patients and provide a reference value for the screening and diagnosis of clinical DM.

Developing a Semi-Supervised Approach Using a PU-Learning-Based Data Augmentation Strategy for Multitarget Drug Discovery.

Multifactorial diseases demand therapeutics that can modulate multiple targets for enhanced safety and efficacy, yet the clinical approval of multitarget drugs remains rare. The integration of machine learning (ML) and deep learning (DL) in drug discovery has revolutionized virtual screening. This study investigates the synergy between ML/DL methodologies, molecular representations, and data augmentation strategies. Notably, we found that SVM can match or even surpass the performance of state-of-the-art DL methods. However, conventional data augmentation often involves a trade-off between the true positive rate and false positive rate. To address this, we introduce Negative-Augmented PU-bagging (NAPU-bagging) SVM, a novel semi-supervised learning framework. By leveraging ensemble SVM classifiers trained on resampled bags containing positive, negative, and unlabeled data, our approach is capable of managing false positive rates while maintaining high recall rates. We applied this method to the identification of multitarget-directed ligands (MTDLs), where high recall rates are critical for compiling a list of interaction candidate compounds. Case studies demonstrate that NAPU-bagging SVM can identify structurally novel MTDL hits for ALK-EGFR with favorable docking scores and binding modes, as well as pan-agonists for dopamine receptors. The NAPU-bagging SVM methodology should serve as a promising avenue to virtual screening, especially for the discovery of MTDLs.

Maximizing Emotion Recognition Accuracy with Ensemble Techniques on EEG Signals

Background:: Emotion is a strong feeling such as love, anger, fear, etc. Emotion can be recognized in two ways, i.e., External expression and Biomedical data-based. Nowadays, various research is occurring on emotion classification with biomedical data. Aim:: One of the most current studies in the medical sector, gaming-based applications, education sector, and many other domains is EEG-based emotion identification. The existing research on emotion recognition was published using models like KNN, RF Ensemble, SVM, CNN, and LSTM on biomedical EEG data. In general, only a few works have been published on ensemble or concatenation models for emotion recognition on EEG data and achieved better results than individual ones or a few machine learning approaches. Various papers have observed that CNN works better than other approaches for extracting features from the dataset, and LSTM works better on the sequence data. Method:: Our research is based on emotion recognition using EEG data, a mixed-model deep learning methodology, and its comparison with a machine learning mixed-model methodology. In this study, we introduced a mixed model using CNN and LSTM that classifies emotions in valence and arousal on the DEAP dataset with 14 channels across 32 people. Result and Discussion: We then compared it to SVM, KNN, and RF Ensemble, and concatenated these models with it. First preprocessed the raw data, then checked emotion classification using SVM, KNN, RF Ensemble, CNN, and LSTM individually. After that with the mixed model of CNN-LSTM, and SVM-KNN-RF Ensemble results are compared. Proposed model results have better accuracy as 80.70% in valence than individual ones with CNN, LSTM, SVM, KNN, RF Ensemble and concatenated models of SVM, KNN and RF Ensemble. Conclusion:: Overall, this paper concludes a powerful technique for processing a range of EEG data is the combination of CNNs and LSTMs. Ensemble approach results show better performance in the case of valence at 80.70% and 78.24% for arousal compared to previous research.

Breast Cancer Prediction Using Support Vector Machine Ensemble with PCA Feature Selection Method

Breast Cancer Prediction Using Support Vector Machine Ensemble with PCA Feature Selection Method

Speech Emotion Recognition Using a Multi-Time-Scale Approach to Feature Aggregation and an Ensemble of SVM Classifiers

Due to its relevant real-life applications, the recognition of emotions from speech signals constitutes a popular research topic. In the traditional methods applied for speech emotion recognition, audio features are typically aggregated using a fixed-duration time window, potentially discarding information conveyed by speech at various signal durations. By contrast, in the proposed method, audio features are aggregated simultaneously using time windows of different lengths (a multi-time-scale approach), hence, potentially better utilizing information carried at phonemic, syllabic, and prosodic levels compared to the traditional approach. A genetic algorithm is employed to optimize the feature extraction procedure. The features aggregated at different time windows are subsequently classified by an ensemble of support vector machine (SVM) classifiers. To enhance the generalization property of the method, a data augmentation technique based on pitch shifting and time stretching is applied. According to the obtained results, the developed method outperforms the traditional one for the selected datasets, demonstrating the benefits of using a multi-time-scale approach to feature aggregation.

Early warning model based on support vector machine ensemble algorithm

Support Vector Machine (SVM) is a powerful machine learning technique often applied in various early warning scenarios. When using SVM for binary classification, its output results can be converted into a probability distribution based on sample categories. Through the analysis of the results, it is found that when the probability of a sample belonging to a positive class is equivalent to the probability of a sample belonging to a negative class, there is a high possibility of misclassification of the sample. To address this issue, this study proposes a new SVM ensemble model construction method. We generate SVM sub-classifiers to extract error prone set, and utilise three different combination rules to construct ensemble model based on the output probability of the samples. The accuracy of the whole data set is improved for the classification accuracy of the easily misclassified data are improved. This research evaluated the model based on Wind turbine SCADA data and UCI benchmark datasets. According to the research results, the accuracy rate of SCADA data is as high as 97.52%, and the accuracy rate of German credit data is 77.50%. Meanwhile, the comparison with other methods also validates the effectiveness of the proposed method.

Integrating classic AI and agriculture: A novel model for predicting insecticide-likeness to enhance efficiency in insecticide development

The integration of artificial intelligence (AI) into smart agriculture boosts production and management efficiency, facilitating sustainable agricultural development. In intensive agricultural management, adopting eco-friendly and effective pesticides is crucial to promote green agricultural practices. However, exploring new insecticides species is a difficult and time-consuming task that involves significant risks. Enhancing compound druggability in the lead discovery phase could considerably shorten the discovery cycle, accelerating insecticides research and development. The Insecticide Activity Prediction (IAPred) model, a novel classic artificial intelligence-based method for evaluating the potential insecticidal activity of unknown functional compounds, is introduced in this study. The IAPred model utilized 27 insecticide-likeness features from PaDEL descriptors and employed an ensemble of Support Vector Machine (SVM) and Random Forest (RF) algorithms using the hard-vote mechanism, achieving an accuracy rate of 86 %. Notably, the IAPred model outperforms current models by accurately predicting the efficacy of novel insecticides such as nicofluprole, overcoming the limitations inherent in existing insecticide structures. Our research presents a practical approach for discovering and optimizing novel insecticide lead compounds quickly and efficiently.

ABES: attention bi-directional ensemble SVM for early detection of brain tumors

ABES: attention bi-directional ensemble SVM for early detection of brain tumors

Optimization of parallel SVM algorithm for big data

Parallel Support Vector Machine (SVM) based on big data has achieved some results in data mining, but due to the complexity of the data itself and a large amount of noisy data, its execution efficiency and classification accuracy in the big data environment are very low. In order to eliminate noise, a noise reduction method based on Noise Cleaning (NC) strategy was proposed, and redundant training samples in big data environments were deleted; Introduce an improved Artificial Fish Swarm Algorithm (IAFSA) to obtain the final Parallel SVM algorithm using mutual information and artificial fish swarm algorithm based on MapReduce (MIAFSA-PSVM) classification model. The results indicate that when compared to CMI-PSVM, the execution time of MIAFSA-PSVM algorithm is higher on the NDC dataset with the largest data size, The SVM parameter optimization algorithm based on MapReduce and cuckoo search (CSSVM-MR) and the particle swarm optimization based parallel support vector machine ensemble algorithm (PSO-PSVM) decreased by 40.1%, 79.3%, and 51.7%, respectively. This indicates that GIESVM-MR and MIAFSA-PSVM have strong adaptability to big data environments and high classification accuracy.

A novel approach for location selection in logistics: Ensemble SVM and improved genetic algorithm

SummaryGathering logistics operations in an individual location offers various benefits at the macro level namely reducing environmental and community issues, replacing overflow of traffic, minimizing air pollution, and more. But the complexity in the selection of location is maximized and the logistic operation chooses the wrong location due to frequent variations in characteristics. Also, it is suitable for capturing long‐term dependencies. To overcome these difficulties, we propose ESVM‐IGS; an Ensemble support vector machine, and an Improved Genetic algorithm, with an initial search strategy to improve the efficiency. The Ensemble SVM is applied to produce a better outcome. To identify optimal configurations in the complex optimization problem an initial search‐based improved genetic algorithm is implemented. To conduct our experiments, the ESVM‐IGS is rigorously evaluated on the GIS real‐time Dataset and the efficiency of the model is validated with various performance measures. From the analysis, the proposed method results that it solved the complexity burden and improved the selection ability of long‐term dependency. The experimental results depict the better efficiency of the ESVM‐IGS method for the location selection strategy of logistics.

An adaptive learning paradigm: event detection through a novel dynamic arithmetic optimization-based ensemble SVM for data stream classification

An adaptive learning paradigm: event detection through a novel dynamic arithmetic optimization-based ensemble SVM for data stream classification

Brain Damage Detection Using Ensemble Classifier for Children Health Progress

Magnetic Resonance Imaging (MRI) is a medical imaging technique used to detect brain damage, particularly in children. Previous classification techniques for brain damage had low accuracy, complex edge detection, and high noise levels. In this proposed work, use an ensemble classification technique to analyze MRI brain images of 6-10 year old children. The input data is preprocessed using a Median filter to remove noise and improve image quality before being sent to a segmentation technique. Each part of the segmented image, such as tissue, inner layer, overall shape, and texture, is classified based on pixel size. The classification is done using an ensemble of Support Vector Machines (SVM) and K-Nearest Neighbors (KNN) techniques. The ensemble prediction is multiplied with several input values to classify the MRI image, aggregating different layer values. The output classified image is validated and developed using Mat Lab software simulation, with an accuracy of 97.86%. This method can accurately classify various types of brain damage and aid in early detection in children.

Intrusion Detection System on Cloud Computing using Ensemble SVM

This study introduces a specialized Intrusion Detection System (IDS) designed specifically for cloud computing environments. By leveraging the UNSW_NB15 dataset, it employs feature selection techniques such as SelectKBest and ANOVA to extract relevant features, thereby improving the overall performance of the model. The IDS framework encompasses data preprocessing, Ensemble SVM model training, and performance evaluation utilizing standard metrics. The key methodology revolves around training SVM models using bagging and boosting techniques on preprocessed data, resulting in resilient intrusion detection models. These models are subsequently utilized to determine whether a given set of input features signifies a network intrusion. Through experimental analysis, the research demonstrates the system's efficacy in accurately detecting network intrusions, shedding light on its robustness and dependability.

Monitoring the after-effects of ischemic stroke through EEG microstates.

Stroke may cause extensive after-effects such as motor function impairments and disorder of consciousness (DoC). Detecting these after-effects of stroke and monitoring their changes are challenging jobs currently undertaken via traditional clinical examinations. These behavioural examinations often take a great deal of manpower and time, thus consuming significant resources. Computer-aided examinations of the electroencephalogram (EEG) microstates derived from bedside EEG monitoring may provide an alternative way to assist medical practitioners in a quick assessment of the after-effects of stroke. In this study, we designed a framework to extract microstate maps and calculate their statistical parameters to input to classifiers to identify DoC in ischemic stroke patients automatically. As the dataset is imbalanced with the minority of patients being DoC, an ensemble of support vector machines (EOSVM) is designed to solve the problem that classifiers always tend to be the majority classes in the classification on an imbalanced dataset. The experimental results show EOSVM get better performance (with accuracy and F1-Score both higher than 89%), improving sensitivity the most, from lower than 60% (SVM and AdaBoost) to higher than 80%. This highlighted the usefulness of the EOSVM-aided DoC detection based on microstates parameters. Therefore, the classifier EOSVM classification based on features of EEG microstates is helpful to medical practitioners in DoC detection with saved resources that would otherwise be consumed in traditional clinic checks.

Abstract 4907: RanBALL: Identifying B-cell acute lymphoblastic leukemia subtypes based on an ensemble random projection model

Abstract As the most common pediatric malignancy, B-cell acute lymphoblastic leukemia (B-ALL) has multiple distinct subtypes characterized by recurrent and sporadic somatic and germline genetic alterations like chromosomal alteration, transcription factor rearrangement or kinase inhibition. The treatment of B-ALL patients is personalized based on specific subtypes, as the treatment responses for different B-ALL subtypes may vary considerably. Identification of B-ALL subtypes can facilitate risk stratification and enable tailored therapeutic approaches. Existing methods for B-ALL subtyping primarily depend on immunophenotypic, cytogenetic and genomic analyses, which would be costly, complicated, and laborious in clinical practice applications. To overcome these challenges, we present RanBALL (an Ensemble Random Projection-Based Model for Identifying B-Cell Acute Lymphoblastic Leukemia Subtypes), an accurate and cost-effective model for B-ALL subtype identification based on transcriptomic profiling only. RanBALL leverages random projection (RP) to construct an ensemble of dimension-reduced multi-class classifiers for B-ALL subtyping. Specifically, the transcriptomic profiling features were projected onto low-dimensional spaces by random projection matrices whose elements conform to a distribution characterized by zero mean and unit variance. To ensure reliable and robust performance, we selected 20 subspace dimensions ranging from 600 to 2500, with intervals of 100. The transformed low dimensional data matrix was used for training an ensemble of multi-class support vector machine (SVM) classifiers, each corresponding to one of the RP matrices of various dimensions. The predicted probabilistic scores of each B-ALL subtype were integrated for determining the final decision. Results based on 10 times 10-fold cross validation tests for >1700 B-ALL patients demonstrated that the proposed model achieved an accuracy of 93.7%, indicating promising prediction capabilities of RanBALL for B-ALL subtyping. Furthermore, the 30% held-out tests suggested that the model was robust and consistent to maintain high confidence levels for accurate predictions. The high accuracies of RanBALL suggested that our model could effectively capture underlying patterns of transcriptomic profiling for accurate B-ALL subtype identification. To extend the impact of RanBALL, we have established a free and publicly available python package for RanBALL available at https://github.com/wan-mlab/RanBALL. We believe RanBALL will facilitate the discovery of B-ALL subtype-specific marker genes and therapeutic targets, and eventually have consequential positive impacts on downstream risk stratification and tailored treatment design. Citation Format: Lusheng Li, Hanyu Xiao, Joseph D. Khoury, Jieqiong Wang, Shibiao Wan. RanBALL: Identifying B-cell acute lymphoblastic leukemia subtypes based on an ensemble random projection model [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2024; Part 1 (Regular Abstracts); 2024 Apr 5-10; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2024;84(6_Suppl):Abstract nr 4907.

ENSEMBLE RESAMPLING SUPPORT VECTOR MACHINE, MULTINOMIAL REGRESSION TO MULTICLASS IMBALANCED DATA

Imbalanced data is a commonly encountered issue in classification analysis. This issue gives rise to prediction errors in the classification process, which in turn affects the sensitivity, particularly in the minority class. Resampling techniques can be employed as a means to mitigate the issue of Imbalanced data. Furthermore, ensemble approaches are Utilized in the classification procedure to augment the performance of classification. The present study assesses the efficacy of the bagging ensemble approach in conjunction with ADASYN as a means of addressing the aforementioned issue. The dataset Utilized in this work comprises Imbalanced Glass Identification data, Imbalanced Iris data, and Imbalanced synthetic data. The study Centres on the Utilization of Support Vector Machines (SVM) with parameter optimization using repeated cross-validation (k = 10) and the application of multinomial regression. The evaluation of classification outcomes involves a comparison between the ensemble technique and multinomial regression. This comparison is conducted under pre- and post-resampling conditions, with the evaluation metrics being accuracy, sensitivity, and specificity. The analysis of classification outcomes across the three datasets suggests that the ensemble resampling SVM approach and multinomial regression exhibit superior performance compared to the ensemble SVM and multinomial regression approaches when applied to non-resampled data. Resampling of data has been observed to enhance sensitivity, particularly in the minority class.

A MENTAL ILLNESS EARLY WARNING ANALYSIS MODEL USING AN IMPROVED SVM ALGORITHM AND EEG SIGNALS

The fast-paced lifestyle, intense competition, and constantly changing environment of modern society are important factors contributing to the emergence of psychological disorders. Factors such as work pressure, family conflicts, and interpersonal relationship difficulties pose challenges to individual mental health. Prolonged use of electronic devices and the widespread use of social media further exacerbate mental health issues, as the constant influx of information makes it difficult for people to relax and focus. The harm of psychological disorders to individuals and society cannot be ignored. It not only diminishes the quality of life for patients but also affects work efficiency and social participation. In extreme cases, psychological disorders can lead to serious consequences like suicide, causing immense pain to families and society. Therefore, prioritizing mental health, raising awareness of mental well-being, and timely intervention and treatment of psychological disorders are crucial for individual happiness and social stability. Based on this, this study employs an improved Support Vector Machine (SVM) and Electroencephalogram (EEG) signals for the diagnosis of psychological disorders, aiming for more effective treatment of mental health conditions. Specifically, the EEG signals of the experimental subjects are first collected as training data for the model. The dataset undergoes preprocessing operations such as cleaning and normalization. Due to the limited availability of published information about individuals with psychological disorders, an ensemble SVM algorithm is used as a warning model to improve the accuracy of psychological disorder diagnosis. This algorithm utilizes a differential measurement method to adjust the selection of SVM parameters. The training sample size remains constant, with several iterations of training performed. By combining the differential measurement method, a specific parameter combination is obtained, maximizing the differentiation between base classifiers. This ultimately addresses issues such as small sample size and insufficient sample information in the dataset. Experimental results demonstrate that the model used in this study exhibits good performance in diagnosing psychological disorders.

Classification of human walking context using a single-point accelerometer

Real-world walking data offers rich insights into a person’s mobility. Yet, daily life variations can alter these patterns, making the data challenging to interpret. As such, it is essential to integrate context for the extraction of meaningful information from real-world movement data. In this work, we leveraged the relationship between the characteristics of a walking bout and context to build a classification algorithm to distinguish between indoor and outdoor walks. We used data from 20 participants wearing an accelerometer on the thigh over a week. Their walking bouts were isolated and labeled using GPS and self-reporting data. We trained and validated two machine learning models, random forest and ensemble Support Vector Machine, using a leave-one-participant-out validation scheme on 15 subjects. The 5 remaining subjects were used as a testing set to choose a final model. The chosen model achieved an accuracy of 0.941, an F1-score of 0.963, and an AUROC of 0.931. This validated model was then used to label the walks from a different dataset with 15 participants wearing the same accelerometer. Finally, we characterized the differences between indoor and outdoor walks using the ensemble of the data. We found that participants walked significantly faster, longer, and more continuously when walking outdoors compared to indoors. These results demonstrate how movement data alone can be used to obtain accurate information on important contextual factors. These factors can then be leveraged to enhance our understanding and interpretation of real-world movement data, providing deeper insights into a person’s health.