Support Vector Machine Ensemble Research Articles

Sentiment analysis of sarcasm detection in social media

When people talk or write more especially on social media, they tend to sometimes, flip the polarity of their expressions. Oftentimes, this change of mood affects the accuracy of the sentiment analysis. Sarcasm is a sophisticated form of irony widely used in social networks and micro blogging websites and is usually a type of sentiment where person speaks the contradictory of what individual word means, expressing gloomy feelings applying positive words. However, it is hard even for humans to recognize. Hence, sarcastic expressions play a vital role in the outcome of automatic sentiment detection in sentiment analysis. To improve on the performance of automatic detection of sentiments in social media networks, there is need for sarcastic statements to be recognised and classify correctly. Sentiment Analysis is a method for identifying people’s opinion, attitude, sentiment, and emotion towards any specific target such as individuals, events, topics, product, organizations, services etc. This paper proposed an ensemble classification method having base classifiers as Random Forest, K-Nearest Neighbour and Naive Bayes for detecting sarcasm using lexical, pragmatic, linguistic incongruity and context incongruity features. The same ensemble method and Support Vector Machines (SVM) were used to identify the sarcasm types so as to diagnose the mood of the users. We report the results and present a comparative evaluation of the ensemble method and SVM classifiers for sarcasm and sarcasm type detection indicating their suitability for the task. The results obtained from the set of experiments conducted indicate the reliability of our approach.

An Efficient Hybrid Ensemble SVM for Optimal Channel and Power Allocation Using Chaotic Quantum Bat Optimization

This paper addresses the challenge of resource allocation in wireless networks, given the increasing usage of mobile devices and sensors. To achieve energy efficiency, we propose a novel method called Hybrid Random Forest Ensemble Support Vector Machine (RFESVM) with Chaotic Cloud Quantum Bat Algorithm (CCQBA). The proposed method identifies the best power allocation for each user in Device-to-Device (D2D) communication, which can significantly improve coverage and lower data rates and latency. To overcome the shortcomings of existing methods, we use the chaotic cloud quantum bat algorithm that combines randomness, ergodicity of chaotic mapping, and stability inclination to enhance the convergence speed. To address the issue of class imbalance in the dataset, we combine SVM with RF and establish an ensemble technique called RFESVM. Our proposed method achieves higher energy efficiency compared to other techniques, demonstrating the effectiveness of optimal power allocation in D2D communication. Overall, the proposed method can significantly improve the systems' energy efficiency while ensuring high network performance in wireless networks.

Probability-weighted ensemble support vector machine for intelligent recognition of moving wear debris from joint implant

Friction-induced wear debris from joint implants are effective resources in investigating artificial joint wear and cellular immune response mechanisms. To improve the accuracy of wear debris analysis, an intelligent recognition method is developed for wear debris measurement under motion conditions. In this method, the multi-view image sequence of moving wear debris is captured to acquire the variations of aspect ratio, area, and roundness features. Then multiple SVM models are integrated to identify wear debris types based on weighted probability to improve the accuracy. The proposed method can achieve a classification accuracy of 90.51%, which is better than HIVE-COTE2.0, MultiRocket, and other time series classification algorithms. This method can be applied to monitor wear status of artificial joint articulating surfaces.

Multivariate variational mode decomposition & phase space reconstruction based motor imagery EEG classification

Electroencephalogram signals have gained much attention in recent years as these signals help in decoding a person's intentions for performing various actions using the Brain Computer Interface system. However, it is challenging to accurately classify human intentions as the electroencephalogram signals are non-linear and non-stationary. This work proposes a novel methodology for feature extraction and classification of motor imagery electroencephalogram signals. The proposed methodology is based on Multivariate Variational Mode Decomposition and Phase Space Reconstruction. A number of statistical and non-linear features are extracted and classified using the Ensemble Support Vector Machine model. The efficiency of the methodology is evaluated on BCI competition-IV(2a) dataset. The evaluation results conclude that the proposed methodology is able to classify both binary and multi-class electroencephalogram data with an average classification accuracy of 96.76% and 84.22%, respectively. Hence, the proposed methodology can be utilized in designing a real-time Brain Computer interface system.

A Cost Sensitive SVM and Neural Network Ensemble Model for Breast Cancer Classification

A Cost Sensitive SVM and Neural Network Ensemble Model for Breast Cancer Classification

Two-Stage PNN–SVM Ensemble for Higher Education Admission Prediction

In this paper, we investigate the methods used to evaluate the admission chances of higher education institutions’ (HEI) entrants as a crucial factor that directly influences the admission efficiency, quality of education results, and future students’ life-long trajectories. Due to the conditions of uncertainty surrounding the decision-making process that determines the admission of entrants and the inability to independently assess the probability of potential outcomes, we propose the application of the machine learning (ML) model as an algorithm that provides decision-making support. The proposed model includes the support vector machine (SVM) stacking ensemble, which expands the input data set obtained using the Probabilistic Neural Network (PNN). The basic algorithms include four SVM ensemble methods with different kernel functions and Logistic Regression (LR) as a meta-algorithm. We evaluate the accuracy of the developed model in three stages: comparison with existing ML methods; comparison with a single-based model that comprises it; and comparison with a similar stacking model and with other types of ensembles (boosting, begging). The results of the designed two-stage PNN–SVM ensemble model provided an accuracy of 94% and possessed acquired superiority in the comparison stages. The obtained results enable the use of the presented model in the subsequent stages of the development of an intellectual support system for decision making regarding entrants’ admission.

Developing A Comprehensive Machine Learning Framework for Accurate Prediction and Cardiovascular Disease Risk

Cardiovascular diseases are one of the most fatal health conditions worldwide, posing a significant burden on national healthcare systems. Several risk factors, including high blood pressure, family history, stress, age, gender, cholesterol, BMI, and an unhealthy lifestyle, have been identified for these diseases. Early diagnosis is crucial for effective treatment, but the accuracy of existing approaches needs improvement due to the life-threatening nature of cardiovascular disorders. This study proposes a MaLCaDD framework for precise prediction of cardiovascular diseases. The methodology addresses missing data and imbalances, using Feature Importance to select relevant features, and an ensemble of Logistic Regression, SVM, Random Forest, Decision Tree, and KNN classifiers for accurate prediction. The study shows that MaLCaDD outperforms current state-of-the-art techniques, making it a trustworthy tool for early detection of cardiovascular diseases. The developed framework will be thoroughly tested and validated using different evaluation metrics, including sensitivity, specificity, and area under the curve, to ensure its accuracy and reliability. The ultimate goal of this project is to provide healthcare practitioners and policymakers with a reliable tool for predicting an individual's cardiovascular disease risk, which can help in early intervention and management, and ultimately reduce the burden of cardiovascular disease on society.

A novel hybrid ensemble learning for anomaly detection in industrial sensor networks and SCADA systems for smart city infrastructures

Critical Infrastructures (CIs) use Supervisory Control and Data Acquisition (SCADA) systems for monitoring and remote control. Sensor networks are being integrated into all areas of the infrastructures of smart cities. The sensor network data stream contains information that can be utilized to model and control the activity of these infrastructures. However, SCADA systems are constantly exposed to a variety of diverse intrusions, making detection with traditional intrusion detection systems (IDS) extremely difficult. Due to their unique specifications, conventional security solutions, like antivirus and firewall software, are unsuitable for properly securing SCADA systems. In addition, anomaly detection in industrial sensor networks (ISNs) should occur in real time. Therefore, effectively identifying cyberattacks in major SCADA systems is unquestionably essential for enhancing their resilience, ensuring safe operations, and avoiding expensive maintenance. We developed a novel hybrid ensemble model approach to address these issues. This paper's primary objective is to detect hostile intrusions that have already circumvented firewalls and typical IDS. In this paper, we propose a hybrid Ensemble Learning Model (ELM) for intrusion detection in SCADA systems with ISNs utilizing a tangible data gathered from a gas pipeline system given by Mississippi State University (MSU), the water system, and the high-dimensional University of New South Wales-NB 2015 (UNSW-NB15) data that reflects a typical attack in the Internet of Things (IoT) environment. The unity normalization method was adopted for data preprocessing, and the Principal Component Analysis (PCA) was utilized for feature extraction of the high-dimensional datasets. Grey Wolf Optimizer (GWO) was used for optimizing the bagging, stacking, Adaboost, and an ensemble of classifiers Naive Bayes and Support Vector Machine with a majority voting technique. Then, we utilized the proposed approach founded on the bijective soft-set approach for efficient ELM selection. The experiment was conducted in two phases: Initially, without PCA + GWO for feature extraction and selection on the ELM, and subsequently, with PCA + GWO for feature extraction and selection on the ELM. PCA + GWO on the ensemble of classifiers NB + SVM provided an accuracy of 99%, precision of 100%, recall of 100%, and detection rate of 99.90%, outpacing the ensemble of classifiers without PCA feature extraction and GWO optimization approaches.

Classification of Multiclass Ensemble SVM for Human Activities based on Sensor Accelerometer and Gyroscope

Human Activity Recognition is technology introduced to recognize human activities. Several technologies that have been applied are Accelerometer sensors, Gyroscope sensors, Cameras, and GPS. The selection of the Support Vector Machine algorithm is due to its capabilities to minimize errors in training data sets and the Curse of dimensionality which can estimate parameters as well as its ability to find the best hyperplane that separates two classes. The SVM algorithm was originally developed for the classification of two classes. Problem raised if there are more than two classes. In addition, the performance will not optimal for the large-scale data. Therefore, modification the current design is needed. An ensemble technique can be used to combine the Support Vector Machine algorithm with the bagging algorithm. This study proposes the application of an ensemble SVM algorithm to classify human activities based on accelerometers and gyroscope sensors on smartphones. The total data is 13725 records with 4575 representatives of each class. From the results of the overall data partition carried out in the calcification process using the ensemble SVM algorithm, the best performance was generated when comparing datasets with 80% training data and 20% test data from a total of 13725 records because it succeeded in increasing accuracy, precision, and sensitivity.

Ensemble of Support Vector Machines for spectral-spatial classification of hyperspectral and multispectral images

Ensemble of Support Vector Machines for spectral-spatial classification of hyperspectral and multispectral images

Multi-view Representation Induced Kernel Ensemble Support Vector Machine

Multi-view Representation Induced Kernel Ensemble Support Vector Machine

Optimization-Driven Machine Learning Approach for the Prediction of Hydrochar Properties from Municipal Solid Waste

Municipal solid waste (MSW) management is an essential element of present-day society. The proper storage and disposal of solid waste is critical to public health, safety, and environmental performance. The direct recovery of MSW into useful energy is a critical task. In addition, the demand for conventional power supplies is high. As a strategy to solve these two problems, the technology to directly convert municipal solid waste into conventional energy to replace fossil fuels has been obtained. The hydrothermal carbonization (HTC) process is a thermochemical conversion process that utilizes heat to convert wet biomass feedstocks into hydrochar. Hydrochar with premium gasoline properties is used for fuel combustion for strength. The properties of fuel hydrochar, including C char (carbon content), HHV (higher heating value), and yield, are mainly based on the properties of the MSW. This study aimed to predict the properties of fuel hydrochar using a machine learning (ML) model. We employed an ensemble support vector machine (E-SVM) as the classifier, which was combined with the slime mode algorithm (SMA) for optimization and developed based on 281 data points. The model was primarily trained and tested on a fusion of three datasets: sewage sludge, leftovers, and cow dung. The proposed ESVM_SMA model achieved an excellent overall performance with an average R2 of 0.94 and RMSE of 2.62.

Breast Cancer Prediction using Machine Learning

Abstract: Breast cancer poses a serious threat to women, as it is highly morbid and lethal. Doctors find it challenging to develop a treatment plan that could increase patient survival time due to the lack of reliable prognosis models. Therefore, figuring out a method that outcome in the least amount of wrongdoings while improving productivity requires time. In the paper, SVM, Logistic Regression, Random Forest, XGBoost, AdaBoost, kNearest Neighbors, Naive Bayes and custom ensemble Classifiers. These algorithms that predict the outcome of breast cancer. The JUPYTER platform is used to conduct all experiments, which are carried out in a simulation environment.

Automatic diagnosis of COVID-19 related respiratory diseases from speech.

In this work, an attempt is made to propose an intelligent and automatic system to recognize COVID-19 related illnesses from mere speech samples by using automatic speech processing techniques. We used a standard crowd-sourced dataset which was collected by the University of Cambridge through a web based application and an android/iPhone app. We worked on cough and breath datasets individually, and also with a combination of both the datasets. We trained the datasets on two sets of features, one consisting of only standard audio features such as spectral and prosodic features and one combining excitation source features with standard audio features extracted, and trained our model on shallow classifiers such as ensemble classifiers and SVM classification methods. Our model has shown better performance on both breath and cough datasets, but the best results in each of the cases was obtained through different combinations of features and classifiers. We got our best result when we used only standard audio features, and combined both cough and breath data. In this case, we achieved an accuracy of 84% and an Area Under Curve (AUC) score of 84%. Intelligent systems have already started to make a mark in medical diagnosis, and this type of study can help better the health system by providing much needed assistance to the health workers.

Performance Evaluation of State-of-the-Art Texture Feature Extraction Techniques on Medical Imagery Tasks

Interpreting medical images is certainly a complex task which requires extensive knowledge [1]. According to Computer Aided Diagnosis (CAD) serves as a second opinion that will help radiologists in diagnosis and on the other hand Content-based Image Retrieval uses visual content to help users browse, search and retrieve similar medical images from a database based on the user’s interest [2-4]. The competency of the CBMIR system depends on feature extraction methods [5]. The textural features are very important to determine the content of a medical image. Textural features provide scenic depth, the spatial distribution of tonal variation, and surface orientation [6]. Therefore, this study seeks to compare and evaluate some of the hand-crafted texture feature extraction techniques in CBMIR. This is to help those concerned in enhancing CBIR systems to make informed decisions concerning the selection of the best textural feature extraction techniques. Since there is no clear indication of which of the various texture feature extraction techniques is best suited for a given performance metric when considering which of the techniques to choose for a particular study in CBMIR systems. The objective of this work, therefore, is to comparatively evaluate the performance of the following texture feature extraction techniques; Local Binary Pattern (LBP), Gabor Filter, Gray-Level Co-occurrence Matrix (GLCM), Haralick Descriptor, Features from Accelerated Segment Test (FAST) and Features from Accelerated Segment Test and Binary Robust Independent Elementary Features (FAST &BRIEF) using the metrics; precision, recall, f1-score, mean squared error (MSE), accuracy and time. These techniques are coupled with specific similarity measure to obtain results. The results showed that LBP, Haralick Descriptor, FAST, and GLCM had the best results in terms of (Precision and Accuracy), Time, F1-Score, and Recall respectively. LBP had 82.05% and 88.23% scores for precision and accuracy respectively. The following scores represent the performance of the Haralick descriptor, FAST, and GLCM models respectively; 0.88s, 38.7%, and 44.82%. These test scores are obtained from datasets ranging from 1k-10.5k. Aside from LBP outperforming the other 5 models mentioned, it still outperformed the following proposed models ' [7]’, ‘Tamura texture feature and wavelet transform combined with Hausdorff distance - [8]’, ‘ [9]’ in terms of (precision, accuracy, and recall) and (precision and recall) respectively and probably f1-score (since f1-score is the weighted average of precision and recall). It is believed that an ensemble of LBP, Haralick descriptors, and Support Vector Machine (SVM) can represent a robust system for both medical image retrieval and classification.

A Novel Ensemble of Support Vector Machines for Improving Medical Data Classification

In recent years, the increasing volume and availability of healthcare and biomedical data are opening up new opportunities for computational methods to enhance healthcare in many hospitals. Medical data classification is regarded as the challenging task to develop intelligent medical decision support systems in hospitals. In this paper, the ensemble approaches based on support vector machines are proposed for classifying medical data. This research’s key contribution is that the ensemble multiple support vector machines use the function kernel in the style of gradient boosting and bagging to produce a more accurate fusion model than the mono-modality models. Extensive experiments have been conducted on forty benchmark medical datasets from the University of California at Irvine machine learning repository. The classification results show that there is a statistically significant difference (p-values < 0.05) between the proposed approaches and the best classification models. In addition, the empirical analysis of forty medical datasets indicated that our models can predict diseases with an accuracy rate of 82.82 and 81.76 percent without feature selection in the preprocessing data stage.

Gas liquid cylindrical cyclone flow regime identification using machine learning combined with experimental mechanism explanation

The flow regimes of GLCC with horizon inlet and a vertical pipe are investigated in experiments, and the velocities and pressure drops data labeled by the corresponding flow regimes are collected. Combined with the flow regimes data of other GLCC positions from other literatures in existence, the gas and liquid superficial velocities and pressure drops are used as the input of the machine learning algorithms respectively which are applied to identify the flow regimes. The choosing of input data types takes the availability of data for practical industry fields into consideration, and the twelve machine learning algorithms are chosen from the classical and popular algorithms in the area of classification, including the typical ensemble models, SVM, KNN, Bayesian Model and MLP. The results of flow regimes identification show that gas and liquid superficial velocities are the ideal type of input data for the flow regimes identification by machine learning. Most of the ensemble models can identify the flow regimes of GLCC by gas and liquid velocities with the accuracy of 0.99 and more. For the pressure drops as the input of each algorithm, it is not the suitable as gas and liquid velocities, and only XGBoost and Bagging Tree can identify the GLCC flow regimes accurately. The success and confusion of each algorithm are analyzed and explained based on the experimental phenomena of flow regimes evolution processes, the flow regimes map, and the principles of algorithms. The applicability and feasibility of each algorithm according to different types of data for GLCC flow regimes identification are proposed.

A machine learning approach to the development and prospective evaluation of a pediatric lung sound classification model

Auscultation, a cost-effective and non-invasive part of physical examination, is essential to diagnose pediatric respiratory disorders. Electronic stethoscopes allow transmission, storage, and analysis of lung sounds. We aimed to develop a machine learning model to classify pediatric respiratory sounds. Lung sounds were digitally recorded during routine physical examinations at a pediatric pulmonology outpatient clinic from July to November 2019 and labeled as normal, crackles, or wheezing. Ensemble support vector machine models were trained and evaluated for four classification tasks (normal vs. abnormal, crackles vs. wheezing, normal vs. crackles, and normal vs. wheezing) using K-fold cross-validation (K = 10). Model performance on a prospective validation set (June to July 2021) was compared with those of pediatricians and non-pediatricians. Total 680 clips were used for training and internal validation. The model accuracies during internal validation for normal vs. abnormal, crackles vs. wheezing, normal vs. crackles, and normal vs. wheezing were 83.68%, 83.67%, 80.94%, and 90.42%, respectively. The prospective validation (n = 90) accuracies were 82.22%, 67.74%, 67.80%, and 81.36%, respectively, which were comparable to pediatrician and non-pediatrician performance. An automated classification model of pediatric lung sounds is feasible and maybe utilized as a screening tool for respiratory disorders in this pandemic era.

Annual forecasting of high‐temperature days in China through grey wolf optimization‐based support vector machine ensemble

AbstractWith the intensification of anthropogenic warming and urbanization, high‐temperature weather poses an enormous threat to socio‐economic and human healthy. However, the studies on annual high‐temperature days forecasting based on machine learning are relatively deficient. This study proposes a support vector machine (SVM) ensemble model based on grey wolf optimization (GWO) to predict annual high‐temperature days in Guangzhou, Shanghai and Beijing of China. Atmospheric circulation indices during 1959–2013 were utilized as inputs to train and validate models. The fivefold cross validation was used to expand the sample data and evaluate the performance of the member and ensemble models. The optimal ensemble model for Guangzhou has the highest average R (0.8939) and the lowest average root mean square error (RMSE; 3.3771), followed by the optimal ensemble models for Beijing (0.8871 and 3.6059) and Shanghai (0.7578 and 3.9968). Furthermore, compared with the typical SVM and optimal member models, the average validation RMSE of the optimal ensemble model was improved by 32.6 and 10.0% for Guangzhou, by 29.8 and 9.1% for Shanghai, and by 41.3 and 15.1% for Beijing, respectively. This study demonstrates that the GWO‐based SVM ensemble model can be a promising tool for annual high‐temperature days forecasting due to the nonlinear fitting power of the SVM, the hyperparameters tuning capability of the GWO algorithm, and the integration ability of ensemble learning.

Meta-Heuristic Optimized Hybrid Wavelet Features for Arrhythmia Classification

The non-invasive evaluation of the heart through EectroCardioGraphy (ECG) has played a key role in detecting heart disease. The analysis of ECG signals requires years of learning and experience to interpret and extract useful information from them. Thus, a computerized system is needed to classify ECG signals with more accurate results effectively. Abnormal heart rhythms are called arrhythmias and cause sudden cardiac deaths. In this work, a Computerized Abnormal Heart Rhythms Detection (CAHRD) system is developed using ECG signals. It consists of four stages; preprocessing, feature extraction, feature optimization and classifier. At first, Pan and Tompkins algorithm is employed to detect the envelope of Q, R and S waves in the preprocessing stage. It uses a recursive filter to eliminate muscle noise, <i>T</i>-wave interference and baseline wander. As the analysis of ECG signal in the spatial domain does not provide a complete description of the signal, the feature extraction involves using frequency contents obtained from multiple wavelet filters; bi-orthogonal, Symlet and Daubechies at different resolution levels in the feature extraction stage. Then, Black Widow Optimization (BWO) is applied to optimize the hybrid wavelet features in the feature optimization stage. Finally, a kernel based Support Vector Machine (SVM) is employed to classify heartbeats into five classes. In SVM, Radial Basis Function (RBF), polynomial and linear kernels are used. A total of ∼15000 ECG signals are obtained from the Massachusetts Institute of Technology-Beth Israel Hospital (MIT-BIH) arrhythmia database for performance evaluation of the proposed CAHRD system. Results show that the proposed CAHRD system proved to be a powerful tool for ECG analysis. It correctly classifies five classes of heartbeats with 99.91% accuracy using an RBF kernel with 2^nd level wavelet coefficients. The CAHRD system achieves an improvement of ∼6% over random projections with the ensemble SVM approach and ∼2% over morphological and ECG segment based features with the RBF classifier.