Convolutional Support Vector Machine Research Articles

A novel automatic detection and classification algorithm for elderly cognitive impairment using CSVM

Alzheimer’s disease (AD) is a degenerative disease with insidious onset and chronic nervous system, which can be roughly classified into three stages: preclinical AD, mild cognitive impairment (MCI), and clinical AD. Given that timely treatment of MCI can delay the course of AD, identifying what stage of the disease is at the time of early diagnosis is critical in clinical practice. Based on the survey data of elderly residents in Changsha community, this paper constructs the cognitive impairment data set and explores the application of the emerging classification algorithm Convolutional Support Vector Machine(CSVM) in identifying elderly cognitive impairment, a small data classification problem. CSVM is an improved Support Vector Machine classification(SVC) algorithm, which uses the convolution filtering idea of the Convolutional Neural Networks(CNN) algorithm to preprocess the nonlinearly separable sample set to improve the classification ability of SVM. In addition, the optimal convolution filter (solution) is found by the Simplified Swarm Optimization(SSO) algorithm and orthogonal array experimental design. A variable update amplitude method is also proposed to optimize the SSO update strategy further. Aiming at the problem of an unbalanced sample set, a stratified sampling method is used to divide and cross-validate the sample data, and a variety of evaluation indicators are used to evaluate the performance of the model. Experimental results show that the data set processed by a specific convolution filter can significantly improve the classification performance of linear SVM. The accurate differentiation of normal cognition(NC), MCI, and AD helps intervene in adjuvant treatment for potential patients timely.

Automated Staging of Diabetic Retinopathy Using Convolutional Support Vector Machine (CSVM) Based on Fundus Image Data

Diabetic Retinopathy (DR) is a complication of diabetes mellitus, which attacks the eyes and often leads to blindness. The number of DR patients is significantly increasing because some people with diabetes are not aware that they have been affected by complications due to chronic diabetes. Some patients complain that the diagnostic process takes a long time and is expensive. So, it is necessary to do early detection automatically using Computer-Aided Diagnosis (CAD). The DR classification process based on these several classes has several steps: preprocessing and classification. Preprocessing consists of resizing and augmenting data, while in the classification process, CSVM method is used. The CSVM method is a combination of CNN and SVM methods so that the feature extraction and classification processes become a single unit. In the CSVM process, the first stage is extracting convolutional features using the existing architecture on CNN. CSVM could overcome the shortcomings of CNN in terms of training time. CSVM succeeded in accelerating the learning process and did not reduce the accuracy of CNN's results in 2 class, 3 class, and 5 class experiments. The best result achieved was at 2 class classification using CSVM with data augmentation which had an accuracy of 98.76% with a time of 8 seconds. On the contrary, CNN with data augmentation only obtained an accuracy of 86.15% with a time of 810 minutes 14 seconds. It can be concluded that CSVM was faster than CNN, and the accuracy obtained was also better to classify DR.

Automated Staging of Diabetic Retinopathy Using Convolutional Support Vector Machine (CSVM) Based on Fundus Image Data

Diabetic Retinopathy (DR) is a complication of diabetes mellitus, which attacks the eyes and often leads to blindness. The number of DR patients is significantly increasing because some people with diabetes are not aware that they have been affected by complications due to chronic diabetes. Some patients complain that the diagnostic process takes a long time and is expensive. So, it is necessary to do early detection automatically using Computer-Aided Diagnosis (CAD). The DR classification process based on these several classes has several steps: preprocessing and classification. Preprocessing consists of resizing and augmenting data, while in the classification process, CSVM method is used. The CSVM method is a combination of CNN and SVM methods so that the feature extraction and classification processes become a single unit. In the CSVM process, the first stage is extracting convolutional features using the existing architecture on CNN. CSVM could overcome the shortcomings of CNN in terms of training time. CSVM succeeded in accelerating the learning process and did not reduce the accuracy of CNN's results in 2 class, 3 class, and 5 class experiments. The best result achieved was at 2 class classification using CSVM with data augmentation which had an accuracy of 98.76% with a time of 8 seconds. On the contrary, CNN with data augmentation only obtained an accuracy of 86.15% with a time of 810 minutes 14 seconds. It can be concluded that CSVM was faster than CNN, and the accuracy obtained was also better to classify DR.

Feed-forward networks using logistic regression and support vector machine for whole-slide breast cancer histopathology image classification

The performance of an image classification depends on the efficiency of the feature learning process. This process is a challenging task that traditionally requires prior knowledge from domain experts. Recently, representation learning was introduced to extract features directly from the raw images without any prior knowledge. Deep learning using a Convolutional Neural Network (CNN) has gained massive attention for performing image classification, as it achieves remarkable accuracy that sometimes exceeds human performance. But this type of network learns features by using a back-propagation approach. This approach requires a huge amount of training data and suffers from the vanishing gradient problem that deteriorates the feature learning. The forward-propagation approach uses predefined filters or filters learned outside the model and applied in a feed-forward manner. This approach is proven to achieve good results with small size labeled datasets. In this work, we investigate the suitability of using two feed-forward methods such as Convolutional Logistic Regression Network (CLR), and Convolutional Support Vector Machine Network for Histopathology Images (CSVM-H). The experiments we have conducted on two small breast cancer datasets (Sultan Qaboos University Hospital (SQUH) and BreaKHis dataset) demonstrate the advantage of using feed-forward approaches over the traditional back-propagation ones. On those datasets, the proposed models CLR and CSVM-H were faster to train and achieved better classification performance than the traditional back-propagation methods (VggNet-16 and ResNet-50) on the SQUH dataset. Importantly, our proposed approach CLR and CSVM-H efficiently learn representations from small amounts of breast cancer whole-slide images and achieve an AUC of 0.83 and 0.84, respectively, on the SQUH dataset. Moreover, the proposed models reduce memory footprint in the classification of Whole-Slide histopathology images since their training time is significantly reduced compared to the traditional CNN on the SQUH and BreaKHis datasets.

Improving Solar Radiation Forecasting Utilizing Data Augmentation Model Generative Adversarial Networks with Convolutional Support Vector Machine (GAN-CSVR)

The accuracy of solar radiation forecasting depends greatly on the quantity and quality of input data. Although deep learning techniques have robust performance, especially when dealing with temporal and spatial features, they are not sufficient because they do not have enough data for training. Therefore, extending a similar climate dataset using an augmentation process will help overcome the issue. This paper proposed a generative adversarial network model with convolutional support vector regression, which is named (GAN-CSVR) that combines a GAN, convolutional neural network, and SVR to augment training data. The proposed model is trained utilizing the Multi-Objective loss function, which combines the mean squared error and binary cross-entropy. The original solar radiation dataset used in the testing is derived from three locations, and the results are evaluated using two scales, namely standard deviation (STD) and cumulative distribution function (CDF). The STD and the average error value of the CDF between the original dataset and the augmented dataset for these three locations are 0.0208, 0.1603, 0.9393, and 7.443981, 4.968554, and 1.495882, respectively. These values show very significant similarity in these two datasets for all locations. The forecasting accuracy findings show that the GAN-CSVR model produced augmented datasets that improved forecasting from 31.77% to 49.86% with respect to RMSE and MAE over the original datasets. This study revealed that the augmented dataset produced by the GAN-CSVR model is reliable because it provides sufficient data for training deep networks.

A multimodal deep learning infused with artificial algae algorithm – An architecture of advanced E-health system for cancer prognosis prediction

Cancer becomes life-threatening when it affects vital organs and their potential to function; therefore, early discovery is crucial for prognosis management. Medical research benefits from the rapid growth of Artificial Intelligence (AI) technology, notably machine learning (ML), deep learning (DL), and reinforcement learning (RL) which has resulted in the development of several approaches and techniques for improving cancer prediction and survival analysis. To improve cancer prognosis prediction and survival analysis, a novel framework that fuses multimodal features via early and late fusion techniques using DL and RL is presented. The approach utilizes an artificial algae algorithm (AAA) to extract the most resilient features from the dataset and build a robust model for cancer prognosis and prediction. The uniqueness of this technique is illustrated by using AAA to extract valuable features from a variety of data modalities infused with Double DEEP Q-NETWORK (DDQN), Convolution eXtreme Gradient Boosting (CNN-XGBOOST), and Convolution Support Vector Machine (CNN-SVM) models for cancer prognostic prediction. The effectiveness of the proposed framework was evaluated using publicly-available datasets: Lower Grade Glioma in the Brain(BRAIN-TCGA), Prostate Cancer, Bladder Cancer, Metastatic Colorectal Cancer (MSKCC), and the METABRIC dataset. The optimum accuracy achieved was 0.99 for the BRAIN-TCGA dataset, surpassing a variety of state-of-the-art techniques. The performance assessment results reveal that the proposed model outperforms currently available cancer prognostic prediction models in terms of predictive performance.

Cuckoo Optimized Convolution Support Vector Machine for Big Health Data Processing

Big health data collection and storing for further analysis is a challenging task because this knowledge is big and has many features. Several cloud-based IoT health providers have been described in the literature previously. Furthermore, there are a number of issues related to time consumed and overall network performance when it comes to big data information. In the existing method, less performed optimization algorithms were used for optimizing the data. In the proposed method, the Chaotic Cuckoo Optimization algorithm was used for feature selection, and Convolutional Support Vector Machine (CSVM) was used. The research presents a method for analyzing healthcare information that uses in future prediction. The major goal is to take a variety of data while improving efficiency and minimizing process time. The suggested method employs a hybrid method that is divided into two stages. In the first stage, it reduces the features by using the Chaotic Cuckoo Optimization algorithm with Levy flight, opposition-based learning, and distributor operator. In the second stage, CSVM is used which combines the benefits of convolutional neural network (CNN) and SVM. The CSVM modifies CNN’s convolution product to learn hidden deep inside data sources. For improved economic flexibility, greater protection, greater analytics with confidentiality, and lower operating cost, the suggested approach is built on fog computing. Overall results of the experiments show that the suggested method can minimize the number of features in the datasets, enhances the accuracy by 82%, and decrease the time of the process.

A New Support Vector Machine Based on Convolution Product

The support vector machine (SVM) and deep learning (e.g., convolutional neural networks (CNNs)) are the two most famous algorithms in small and big data, respectively. Nonetheless, smaller datasets may be very important, costly, and not easy to obtain in a short time. This paper proposes a novel convolutional SVM (CSVM) that has the advantages of both CNN and SVM to improve the accuracy and effectiveness of mining smaller datasets. The proposed CSVM adapts the convolution product from CNN to learn new information hidden deeply in the datasets. In addition, it uses a modified simplified swarm optimization (SSO) to help train the CSVM to update classifiers, and then the traditional SVM is implemented as the fitness for the SSO to estimate the accuracy. To evaluate the performance of the proposed CSVM, experiments were conducted to test five well-known benchmark databases for the classification problem. Numerical experiments compared favorably with those obtained using SVM, 3-layer artificial NN (ANN), and 4-layer ANN. The results of these experiments verify that the proposed CSVM with the proposed SSO can effectively increase classification accuracy.

Epilepsy EEG Classification Based on Convolution Support Vector Machine

Clinical Electroencephalogram (EEG) data is of great significance to realize automatable detection, recognition and diagnosis to reduce the valuable diagnosis time. To make a classification of epilepsy, we constructed convolution support vector machine (CSVM) by integrating the advantages of convolutional neural networks (CNN) and support vector machine (SVM). To distinguish the focal and non-focal epilepsy EEG signals, we firstly reduced the dimensionality of EEG signals by using principal component analysis (PCA). After that, we classified the epilepsy EEG signals by the CSVM. The accuracy, sensitivity and specificity of our method reach up to 99.56%, 99.72% and 99.52% respectively, which are competitive than the widely acceptable algorithms. The proposed automatic end to end epilepsy EEG signals classification algorithm provides a better reference for clinical epilepsy diagnosis.

Convolutional Support Vector Models: Prediction of Coronavirus Disease Using Chest X-rays

The disease caused by the new coronavirus (COVID-19) has been plaguing the world for months and the number of cases are growing more rapidly as the days go by. Therefore, finding a way to identify who has the causative virus is impressive, in order to find a way to stop its proliferation. In this paper, a complete and applied study of convolutional support machines will be presented to classify patients infected with COVID-19 using X-ray data and comparing them with traditional convolutional neural network (CNN). Based on the fitted models, it was possible to observe that the convolutional support vector machine with the polynomial kernel (CSVMPol) has a better predictive performance. In addition to the results obtained based on real images, the behavior of the models studied was observed through simulated images, where it was possible to observe the advantages of support vector machine (SVM) models.

CSVM Architectures for Pixel-Wise Object Detection in High-Resolution Remote Sensing Images

Detecting objects becomes an increasingly important task in very high resolution (VHR) remote sensing imagery analysis. With the development of GPU-computing capability, a growing number of deep convolutional neural networks (CNNs) have been designed to address the object detection challenge. However, compared with CPU, GPU is much more costly. Therefore, GPU-based methods are less attractive in practical applications. In this article, we propose a CPU-based method that is based on convolutional support vector machines (CSVMs) to address the object detection challenge in VHR images. Experiments are conducted on three VHR and two unmanned aerial vehicle (UAV) data sets with very limited training data. Results show that the proposed CSVM achieves competitive performance compared to U-Net which is an efficient CNN-based model designed for small training data sets.

Severity Classification of Non-Proliferative Diabetic Retinopathy Using Convolutional Support Vector Machine

Severity Classification of Non-Proliferative Diabetic Retinopathy Using Convolutional Support Vector Machine

GPR B scan image analysis with deep learning methods

Abstract In this paper, we propose a Convolutional Support Vector Machine (CSVM) network for the analysis of Ground Penetrating Radar B Scan (GPR B Scan) images. Similar to a Convolutional Neural Network (CNN) architecture, a CSVM is also a cascade of convolution and pooling layers. However, the main difference is that it utilizes linear Support Vector Machine (SVM) based filters to generate feature maps and follows a forward learning strategy. We applied proposed method for the classification of buried objects, shape type and soil type. We used simulated GPR B scan images to train the networks. Proposed method was tested on both simulated and real GPR B scan images. In addition, we conducted a comparative study of the CSVM method with the classical machine learning approaches and pre-trained CNN models. Experimental results show that the proposed method provides an improved classification performance while the computational complexity is low.

Evaluation of Deep Learning Techniques for Deforestation Detection in the Brazilian Amazon and Cerrado Biomes From Remote Sensing Imagery

Deforestation is one of the major threats to natural ecosystems. This process has a substantial contribution to climate change and biodiversity reduction. Therefore, the monitoring and early detection of deforestation is an essential process for preservation. Techniques based on satellite images are among the most attractive options for this application. However, many approaches involve some human intervention or are dependent on a manually selected threshold to identify regions that suffer deforestation. Motivated by this scenario, the present work evaluates Deep Learning-based strategies for automatic deforestation detection, namely, Early Fusion (EF), Siamese Network (SN), and Convolutional Support Vector Machine (CSVM) as well as Support Vector Machine (SVM), used as the baseline. The target areas are two regions with different deforestation patterns: the Amazon and Cerrado biomes in Brazil. The experiments used two co-registered Landsat 8 images acquired at different dates. The strategies based on Deep Learning achieved the best performance in our analysis in comparison with the baseline, with SN and EF superior to CSVM and SVM. In the same way, a reduction of the salt-and-pepper effect in the generated probabilistic change maps was noticed as the number of training samples increased. Finally, the work assesses how the methods can reduce the time invested in the visual inspection of deforested areas.

Detection of COVID-19 chest X-ray using support vector machine and convolutional neural network

This study aims to detect whether patients examined are healthy, Coronavirus positive, or just have pneumonia based on chest X-ray data using Convolutional Neural Network method as feature extraction and Support Vector Machine as a classification method or called Convolutional Support Vector Machine. Experiments carried out were comparing the kernel used, feature selection methods, architecture in feature extraction, and separated classes. Our instrument reached the accuracy of 97.33% in the separation of 3 classes (normal, pneumonia, COVID19) and 100% in the separation of 2 classes, that is (normal, COVID19) and (pneumonia, COVID19), respectively. Based on these results, it can be concluded that the feature selection method can improve gained accuracy ±98%.

Scene Description for Visually Impaired People with Multi-Label Convolutional SVM Networks

In this paper, we present a portable camera-based method for helping visually impaired (VI) people to recognize multiple objects in images. This method relies on a novel multi-label convolutional support vector machine (CSVM) network for coarse description of images. The core idea of CSVM is to use a set of linear SVMs as filter banks for feature map generation. During the training phase, the weights of the SVM filters are obtained using a forward-supervised learning strategy unlike the backpropagation algorithm used in standard convolutional neural networks (CNNs). To handle multi-label detection, we introduce a multi-branch CSVM architecture, where each branch will be used for detecting one object in the image. This architecture exploits the correlation between the objects present in the image by means of an opportune fusion mechanism of the intermediate outputs provided by the convolution layers of each branch. The high-level reasoning of the network is done through binary classification SVMs for predicting the presence/absence of objects in the image. The experiments obtained on two indoor datasets and one outdoor dataset acquired from a portable camera mounted on a lightweight shield worn by the user, and connected via a USB wire to a laptop processing unit are reported and discussed.

Convolutional support vector machines for speech recognition

Convolutional neural networks (CNNs) have demonstrated the state-of-the-art performances on automatic speech recognition. Softmax activation function for prediction and minimizing the cross-entropy loss is employed by most of the CNNs. This paper proposes a new deep architecture in which two heterogeneous classification techniques named as CNN and support vector machines (SVMs) are combined together. In this proposed model, features are learned using convolution layer and classified by SVMs. The last layer of CNN i.e. softmax layer is replaced by SVMs to efficiently deal with high dimensional features. This model should be interpreted as a special form of structured SVM and named as convolutional support vector machine (CSVM). Instead of training each component separately, the parameters of CNN and SVMs are jointly trained using frame level max-margin, sequence level max-margin, and state-level minimum Bayes risk criterion. The performance of CSVM is checked on TIMIT and Wall Street Journal datasets for phone recognition. By incorporating the features of both CNN and SVMs, CSVM improves the result by 13.33% and 2.31% over baseline CNN and segmental recurrent neural networks respectively.

Wind speed prediction model using singular spectrum analysis, empirical mode decomposition and convolutional support vector machine

Accurate wind speed forecasting is critical to the exploitation and utilization of wind energy. In this paper, a novel wind speed multi-step prediction model is designed based on the SSA (Singular Spectrum Analysis), EMD (Empirical Mode Decomposition) and CNNSVM (Convolutional Support Vector Machine). In the SSA-EMD-CNNSVM model, the SSA is used to reduce the noise and extract the trend information of the original wind speed data; the EMD is used to extract the fluctuation features of the wind speed data and decompose the wind speed time series into a number of sub-layers; and the CNNSVM is used to predict each of the wind speed sub-layers. To investigate the prediction performance of the proposed model, some models are used as the comparison models, including the SVM model, CNNSVM model, EMD-BP model, EMD-RBF model and EMD-Elman model. According to the prediction results of the four experiments, it can be found that the proposed model can have significantly better performance than the seven comparison models from 1-step to 3-step wind speed predictions with the MAPE of 42.85% average performance promotion, MAE of 39.21% average performance promotion, RMSE of 39.25% average performance promotion.

Convolutional SVM Networks for Object Detection in UAV Imagery

Nowadays, unmanned aerial vehicles (UAVs) are viewed as effective acquisition platforms for several civilian applications. They can acquire images with an extremely high level of spatial detail compared to standard remote sensing platforms. However, these images are highly affected by illumination, rotation, and scale changes, which further increases the complexity of analysis compared to those obtained using standard remote sensing platforms. In this paper, we introduce a novel convolutional support vector machine (CSVM) network for the analysis of this type of imagery. Basically, the CSVM network is based on several alternating convolutional and reduction layers ended by a linear SVM classification layer. The convolutional layers in CSVM rely on a set of linear SVMs as filter banks for feature map generation. During the learning phase, the weights of the SVM filters are computed through a forward supervised learning strategy unlike the backpropagation algorithm widely used in standard convolutional neural networks (CNNs). This makes the proposed CSVM particularly suitable for detecting problems characterized by very limited training sample availability. The experiments carried out on two UAV data sets related to vehicles and solar-panel detection issues, with a 2-cm resolution, confirm the promising capability of the proposed CSVM network compared to recent state-of-the-art solutions based on pretrained CNNs.

A Learning Framework for Size and Type Independent Transient Stability Prediction of Power System Using Twin Convolutional Support Vector Machine

Real-time transient stability assessment (TSA) of power systems is an important real world problem in electrical energy engineering and pattern recognition scope. The definition of most discriminative trajectory features and proper supervised trajectory-based classifier has remained a motivational challenge for scholars vis-a-vis real-time TSA. In addition, increase in the consumption of electrical energy along with constraints such as amortization of network equipment induces electric power system inadequacy risk. The retrieval of power system adequacy involves network expansion planning such as installing new power plants for the network. This policy affects the structure and electrical specification of the network significantly. Furthermore, due to sudden or the scheduled tripping of network equipment stemming from action of protection devices or maintenance procedures, the network must undergo shallow structural changes. The different level of changes in network specification is becoming a potential barrier for network analysis tools like real-time TSA platform. In fact, the lack of consideration of the incompatibility of TSA tool with expansion planning affects the performance of TSA learning model that is trained using the pre-expansion network. However, this paradoxical problem can be solved by generalized learning for power system size & type independent (PSs&tInd) real-time TSA. For this purpose, first, we used a set of PSs&tInd trajectory features. Next, we presented a trajectory-based deep neuro classifier to eliminate kernel functions weaknesses plugged into the hyperplane-based classifier. Finally, experimental comparisons were conducted to assess the efficacy of the proposed framework. The results showed that the proposed technique offered high-generalization capacity on real-time TSA during network expansion.