Local Support Vector Machine Research Articles

Optimization of RBF-SVM hyperparameters using genetic algorithm for face recognit

Manual grid-search tuning of machine learning hyperparameters is very time-consuming. Hence, to curb this problem, we propose the use of a genetic algorithm (GA) for the selection of optimal radial-basis-function based support vector machine (RBF-SVM) hyperparameters; regularization parameter C and cost-factor γ. The resulting optimal parameters were used during the training of face recognition models. To train the models, we independently extracted features from the ORL face image dataset using local binary patterns (handcrafted) and deep learning architectures (pretrained variants of VGGNet). The resulting features were passed as input to either linear-SVM or optimized RBF-SVM. The results show that the models from optimized RBFSVM combined with deep learning or hand-crafted features yielded performances that surpass models obtained from Linear-SVM combined with the aforementioned features in most of the data splits. The study demonstrated that it is profitable to optimize the hyperparameters of an SVM to obtain the best classification performance.
 Keywords: Face Recognition, Feature Extraction, Local Binary Patterns, Transfer Learning, Genetic Algorithm and Support Vector Machines.

Local Similarity-Based Fuzzy Multiple Kernel One-Class Support Vector Machine

One-class support vector machine (OCSVM) is one of the most popular algorithms in the one-class classification problem, but it has one obvious disadvantage: it is sensitive to noise. In order to solve this problem, the fuzzy membership degree is introduced into OCSVM, which makes the samples with different importance have different influences on the determination of classification hyperplane and enhances the robustness. In this paper, a new calculation method of membership degree is proposed and introduced into the fuzzy multiple kernel OCSVM (FMKOCSVM). The combined kernel is used to measure the local similarity between samples, and then, the importance of samples is determined based on the local similarity between training samples, so as to determine the membership degree and reduce the impact of noise. The proposed membership requires only positive data in the calculation process, which is consistent with the training set of OCSVM. In this method, the noise has a smaller membership value, which can reduce the negative impact of noise on the classification boundary. Simultaneously, this method of calculating membership has a higher efficiency. The experimental results show that FMKOCSVM based on proposed local similarity membership is efficient and more robust to outliers than the ordinary multiple kernel OCSVMs.

Klasifikasi Sampah Daur Ulang Menggunakan Support Vector Machine Dengan Fitur Local Binary Pattern

Garbage is one of the problems that always arise in Indonesia and even in the world. Increasingly, the production of waste is increased along with the increase in population and consumption. Therefore, need a prevention to stop wasting or producing garbage through recycle. This research do garbage recycle classification of cardboard, glass, metal, paper and plastic by using Local Binary Pattern (LBP) texture feature extraction methode and Support Vector Machine (SVM) as classification methode. For examination technic and dataset distribution is using K-Fold Cross Validation methode type Leave One Out (LOO). From examination result had been done were using fold 5 until fold 10. Polynomial kernel get highest accuracy result from every fold used with mean point 87.82%. Based on SVM classification examination result whether linear kernel, polynomial nor gaussian by using fold 5 until fold 10. The best accuracy point for cardboard garbage is 96.01%. For glass garbage, the best accuracy point is 90.62%. Then, metal garbage get the best accuracy point 89.72%. While paper garbage with highest accuracy point 96.01%. And plastic garbage with highest accuracy point 87.64%.

Proximally sensed digital data library to predict topsoil clay across multiple sugarcane fields of Australia: Applicability of local and universal support vector machine

Knowledge about topsoil (0–0.3 m) clay is required to maintain sugarcane profitability in Queensland, Australia. However, laboratory analysis to get this knowledge is tedious and time consuming. To add value to limited clay data, a digital soil map (DSM) can be created by using digital data and mathematical models. At the field level, site-specific linear regression (LR) models are often used along with gamma-ray (γ-ray) spectrometry and electromagnetic induction data (i.e. soil apparent electrical conductivity – ECa). But these LR might not perform well in site-independent calibrations across multiple sites. In this regard, support vector machine (SVM) might be useful. In this research, we first aimed to determine, using a stepwise SVM and calibration dataset, the optimal digital data (i.e. individual or combined) to develop local (for each individual site) and universal (for combined sites) SVM models. Using optimal digital data, our second aim was to predict clay for validation datasets by using local SVM in site-specific approach and universal SVM in site-independent, holdout and spiking approaches. Using these approaches, DSM of predicted clay and associated uncertainty were generated for a representative study site (i.e., Mossman). The third aim was to determine the suitable number of spiking samples and by varying the size of both spiking set and calibration model. Approaches were compared using prediction agreement (Lin’s concordance) and accuracy (ratio of performance to deviation – RPD). We concluded from stepwise SVM that combining digital data resulted in better accuracy (RPD = 2.17) than individual γ-ray (1.79) or ECa (1.49) data. In terms of independent validation, the results of Mossman reflected the general rank order of different approaches with site-specific (3.03) excellent, spiking (1.89) very good, site-independent (1.84) good and holdout (1.18) poor predictions. In case of DSM and uncertainty maps, under-predictions were problematic at field edges and where digital data changed as a function of soil type and or proximity to a prior stream channel. Considering the suitable number of spiking samples, size of spiking set showed linear relationship with improvement in predictions while that of calibration model showed no influence. These results suggest the usefulness of proximally sensed data and universal SVM to effectively predict clay across multiple sites.

Deep learning approaches for COVID-19 detection based on chest X-ray images

COVID-19 is a novel virus that causes infection in both the upper respiratory tract and the lungs. The numbers of cases and deaths have increased on a daily basis on the scale of a global pandemic. Chest X-ray images have proven useful for monitoring various lung diseases and have recently been used to monitor the COVID-19 disease. In this paper, deep-learning-based approaches, namely deep feature extraction, fine-tuning of pretrained convolutional neural networks (CNN), and end-to-end training of a developed CNN model, have been used in order to classify COVID-19 and normal (healthy) chest X-ray images. For deep feature extraction, pretrained deep CNN models (ResNet18, ResNet50, ResNet101, VGG16, and VGG19) were used. For classification of the deep features, the Support Vector Machines (SVM) classifier was used with various kernel functions, namely Linear, Quadratic, Cubic, and Gaussian. The aforementioned pretrained deep CNN models were also used for the fine-tuning procedure. A new CNN model is proposed in this study with end-to-end training. A dataset containing 180 COVID-19 and 200 normal (healthy) chest X-ray images was used in the study’s experimentation. Classification accuracy was used as the performance measurement of the study. The experimental works reveal that deep learning shows potential in the detection of COVID-19 based on chest X-ray images. The deep features extracted from the ResNet50 model and SVM classifier with the Linear kernel function produced a 94.7% accuracy score, which was the highest among all the obtained results. The achievement of the fine-tuned ResNet50 model was found to be 92.6%, whilst end-to-end training of the developed CNN model produced a 91.6% result. Various local texture descriptors and SVM classifications were also used for performance comparison with alternative deep approaches; the results of which showed the deep approaches to be quite efficient when compared to the local texture descriptors in the detection of COVID-19 based on chest X-ray images.

A smart approach for fire prediction under uncertain conditions using machine learning

One of the most ubiquitous cause of worldwide deforestation and devastation of wildlife is fire. To control fire and reach the forest area in time is not always possible. Consequently, the level of destruction is often high. Therefore, predicting fires well in time and taking immediate action is of utmost importance. However, traditional fire prediction approaches often fail to detect fire in time. Therefore, a more reliable approach like the Internet of Things (IoT) needs to be adopted. IoT sensors can not only observe the real-time conditions of an area, but it can also predict fire when combined with Machine learning. This paper provides an insight into the use of Machine Learning models towards the occurrence of forest fires. In this context, eight Machine Learning algorithms: Boosted Decision Trees, Decision Forest Classifier, Decision Jungle Classifier, Averaged Perceptron, 2-Class Bayes Point Machine, Local Deep Support Vector Machine (SVM), Logistic Regression and Binary Neural Network model have been implemented. Results suggest that the Boosted decision tree model with the Area Under Curve (AUC) value of 0.78 is the most suitable candidate for a fire prediction model. Based on the results, we propose a novel IoT-based smart Fire prediction system that would consider both meteorological data and images for early fire prediction.

Detection of coronavirus Disease (COVID-19) based on Deep Features and Support Vector Machine

The detection of coronavirus (COVID-19) is now a critical task for the medical practitioner. The coronavirus spread so quickly between people and approaches 100,000 people worldwide. In this consequence, it is very much essential to identify the infected people so that prevention of spread can be taken. In this paper, the deep feature plus support vector machine (SVM) based methodology is suggested for detection of coronavirus infected patient using X-ray images. For classification, SVM is used instead of deep learning based classifier, as the later one need a large dataset for training and validation. The deep features from the fully connected layer of CNN model are extracted and fed to SVM for classification purpose. The SVM classifies the corona affected X-ray images from others. The methodology consists of three categories of Xray images, i.e., COVID-19, pneumonia and normal. The method is beneficial for the medical practitioner to classify among the COVID-19 patient, pneumonia patient and healthy people. SVM is evaluated for detection of COVID-19 using the deep features of different 13 number of CNN models. The SVM produced the best results using the deep feature of ResNet50. The classification model, i.e. ResNet50 plus SVM achieved accuracy, sensitivity, FPR and F1 score of 95.33%,95.33%,2.33% and 95.34% respectively for detection of COVID-19 (ignoring SARS, MERS and ARDS). Again, the highest accuracy achieved by ResNet50 plus SVM is 98.66%. The result is based on the Xray images available in the repository of GitHub and Kaggle. As the data set is in hundreds, the classification based on SVM is more robust compared to the transfer learning approach. Also, a comparison analysis of other traditional classification method is carried out. The traditional methods are local binary patterns (LBP) plus SVM, histogram of oriented gradients (HOG) plus SVM and Gray Level Co-occurrence Matrix (GLCM) plus SVM. In traditional image classification method, LBP plus SVM achieved 93.4% of accuracy.

Lithofacies identification using support vector machine based on local deep multi-kernel learning

Lithofacies identification is a crucial work in reservoir characterization and modeling. The vast inter-well area can be supplemented by facies identification of seismic data. However, the relationship between lithofacies and seismic information that is affected by many factors is complicated. Machine learning has received extensive attention in recent years, among which support vector machine (SVM) is a potential method for lithofacies classification. Lithofacies classification involves identifying various types of lithofacies and is generally a nonlinear problem, which needs to be solved by means of the kernel function. Multi-kernel learning SVM is one of the main tools for solving the nonlinear problem about multi-classification. However, it is very difficult to determine the kernel function and the parameters, which is restricted by human factors. Besides, its computational efficiency is low. A lithofacies classification method based on local deep multi-kernel learning support vector machine (LDMKL-SVM) that can consider low-dimensional global features and high-dimensional local features is developed. The method can automatically learn parameters of kernel function and SVM to build a relationship between lithofacies and seismic elastic information. The calculation speed will be expedited at no cost with respect to discriminant accuracy for multi-class lithofacies identification. Both the model data test results and the field data application results certify advantages of the method. This contribution offers an effective method for lithofacies recognition and reservoir prediction by using SVM.

Deep feature based rice leaf disease identification using support vector machine

Features are the vital factor for image classification in the field of machine learning. The advancement of deep convolutional neural network (CNN) shows the way for identification of rice diseases using deep features with the expectation of high returns. This paper introduced 5932 on-field images of four types of rice leaf diseases, namely bacterial blight, blast, brown spot and tungro. In addition, the performance evaluation of 11 CNN models in transfer learning approach and deep feature plus support vector machine (SVM) was carried out. The simulation results show the deep feature plus SVM perform better classification compared to transfer learning counterpart. Also, the performance of small CNN models such as mobilenetv2 and shufflenet was examined. The performance evaluation was carried out in terms of accuracy, sensitivity, specificity, false positive rate (FPR), F1 Score and training time. Again, the statistical analysis was performed to choose the better classification model. The deep feature of ResNet50 plus SVM performs better with F1 score of 0.9838. The fc6 layer of vgg16, vgg19 and AlexNet have better contribution towards classification compared to fc7 and fc8. Further, the F1 score of CNN classification models was compared with other traditional image classification models such as bag-of-feature, local binary patterns (LBP) plus SVM, histogram of oriented gradients (HOG) plus SVM and Gray Level Co-occurrence Matrix (GLCM) plus SVM.

Fuzzy quasi-linear SVM classifier: Design and analysis

Multiple Support Vector Machine (SVM) classifier based on ensemble learning approaches could be enhanced from the view point of accuracy, but the performance of these classifiers closely depends on the initial condition of the partitioning method used in the design. Furthermore, these classifiers are more easily affected by noise and outliers. In this study, a novel fuzzy quasi-linear SVM classifier realized with the aid of a composite kernel function and Fuzzy C-Means (FCM) clustering is proposed. The objective of this approach is to reduce the effect of noise and outliers and also handle the overfitting problem through the synergistic effect of the two methods: First, Fuzzy C-Means (FCM) is used to partition the training dataset into several subsets as a preprocessing phase of the proposed classifier. Second, the composite kernel based on multiple linear kernel expression is considered to avoid overfitting problem. In more detail, each training data is assigned to the corresponding membership degree. Some data which are potential noise or outliers are assigned with lower membership degrees and thus yield a small contribution to the composite kernel function. Then, the composite kernel function for multiple local SVMs is constructed according to the distribution of training data. The designed fuzzy quasi-linear SVM classifier is tested with both artificial and UCI data sets. It is also applied for sorting the problem of black plastic wastes being handled in the practice in order to verify the effective as well as efficient classification improvement. Experimental results demonstrate that the proposed method shows the elevated classification performance when compared to performance produced by the methods studied previously.

Image Copy Detection Based on Local Binary Pattern and SVM Classifier

Abstract Due to the availability of a large number of image editing software, it is very easy to find duplicate copies of original images. In such a situation, there is a need to develop a robust technique that can be used for the identification of duplicate copies apart from differentiating it from different images. In this paper, we have proposed an image hashing technique based on uniform Local Binary Pattern (LBP). Here, the input image is initially pre-processed before calculating the Local Binary Pattern (LBP) which is used for image identification. Experiments show that proposed hashing gives excellent performance against the Histogram equalization attack. The Receiver Operating Curve (ROC) indicates that the proposed hashing also performs better in terms of robustness and discrimination. Support Vector Machine (SVM) classifier shows that generated features can easily classify images into a set of similar and different images, and can classify new data with a high level of accuracy.

Multiview Feature-Based Sea Surface Small Target Detection in Short Observation Time

In this letter, a small target detection method in sea clutter background is proposed based on multiview polarization features and localized one-class support vector machine (LOCSVM). In the proposed method, three views of features that represent the divergence between the small target and the clutter are extracted: 1) the study of the target scattering characteristics shows that the dominant scattering components of the small target cell (TC) are sphere scattering and diplane scattering, while the scattering mechanism of the clutter cell is varied and affected by the sea conditions. Therefore, the relative powers of the sphere and diplane scattering components are extracted based on the polarimetric scattering matrix; 2) Because the target is a definite shaped object, its cell does not exist in the fractal characteristics, while the clutter cell is on the contrary. Therefore, the average Tsallis entropy (TE) is extracted from full-polarization channels to reflect the fractal characteristics of the cell; and 3) As the generalization of the Doppler entropy, because the frequency spectrum of the TC is more congregate, the TE can reflect the Doppler characteristics. Then, in the detection stage, a novel one-class classifier, referred to as LOCSVM, is designed as the detector. LOCSVM is a combination of <inline-formula xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink"> <tex-math notation="LaTeX">$k$ </tex-math></inline-formula> -means clustering and OCSVM and can enhance the detection performance of OCSVM. The experimental results on the IPIX data show the effectiveness of the proposed method, especially in the case of short observation time.

Short Term Traffic State Prediction via Hyperparameter Optimization Based Classifiers.

Short-term traffic state prediction has become an integral component of an advanced traveler information system (ATIS) in intelligent transportation systems (ITS). Accurate modeling and short-term traffic prediction are quite challenging due to its intricate characteristics, stochastic, and dynamic traffic processes. Existing works in this area follow different modeling approaches that are focused to fit speed, density, or the volume data. However, the accuracy of such modeling approaches has been frequently questioned, thereby traffic state prediction over the short-term from such methods inflicts an overfitting issue. We address this issue to accurately model short-term future traffic state prediction using state-of-the-art models via hyperparameter optimization. To do so, we focused on different machine learning classifiers such as local deep support vector machine (LD-SVM), decision jungles, multi-layers perceptron (MLP), and CN2 rule induction. Moreover, traffic states are evaluated using traffic attributes such as level of service (LOS) horizons and simple if–then rules at different time intervals. Our findings show that hyperparameter optimization via random sweep yielded superior results. The overall prediction performances obtained an average improvement by over 95%, such that the decision jungle and LD-SVM achieved an accuracy of 0.982 and 0.975, respectively. The experimental results show the robustness and superior performances of decision jungles (DJ) over other methods.

Hyperspectral anomaly detection by local joint subspace process and support vector machine

ABSTRACTHyperspectral anomaly detection is a challenging task due to the inaccurate evaluation of background statistics and the contamination of anomaly pixels. In this paper, we propose an effective hyperspectral anomaly detection algorithm based on local joint subspace process and support vector machine (SVM). The method mainly consists of three steps. At first, in the local joint subspace process, we combine Mahalanobis distance detection and spectral angular distance detection of subspace projection by local sliding change dual windows. With this local joint subspace process, the spatial-spectral information is jointly utilized and the local subspace score is also obtained. Then, to further improve the detection probability, we propose the support vector score by classifying the dimensionally reduced hyperspectral image data through a trained SVM model. Finally, we obtain the final detection result from the linear combination of above scores. Compared with alternative methods, the proposed method has shown a superior performance on both synthetic and real-world datasets.

A Hybrid Global Local Adaptive Particle Swarm Optimization-Based Support Vector Machine Model for Human Facial Authentication

A new facial authentication model called global local adaptive particle swarm optimization-based support vector machine, was proposed in this paper. The proposed model aimed to solve the problem of finding the preeminent parameters of support vector machine in order to come out with a powerful human facial authentication technique. The conventional particle swarm optimization algorithm was utilized with support vector machine to explore the preeminent parameters of support vector machine. However, the particle swarm optimization support vector machine model has some limitations in selecting the velocity coefficient and inertia weight. One of the best approaches, which is used to solve the velocity coefficient problem, is adaptive acceleration particle swarm optimization. Also, the global-local best inertia weight is used efficiently for selecting the inertia weight. Therefore, the global local adaptive particle swarm optimization-based support vector machine model was proposed based on combining adaptive acceleration particle swarm optimization, global-local best inertia weight, and support vector machine. The proposed model used the principal component analysis approach for feature extraction, as well as global local adaptive particle swarm optimization for finding the preeminent parameters of support vector machine. In the experiments, two datasets (YALEB and CASIAV5) were used, and the suggested model was compared with particle swarm optimization support vector machine and adaptive acceleration particle swarm optimization support vector machine methods. The comparison was via accuracy, computational time, and optimal parameters of support vector machine. Our model can be used for security applications and apply for human facial authentication.

Multi-class Image Classification Using Deep Learning Algorithm

Classifying images is a complex problem in the field of computer vision. The deep learning algorithm is a computerized model simulates the human brain functions and operations. Training the deep learning model is a costly process in machine resources and time. Investigating the performance of the deep learning algorithm is mostly needed. The convolutional neural network (CNN) is most commonly used to build a structure of the deep learning models. In this paper convolutional neural network (CNN) model pre-trained on Image-Net is used for classification of images of the PASCAL VOC 2007 data-set. The transfer learning approach is used to improve the performance of the deep learning CNN model where classification works fairly well with the smallest amount of computation time and fewer machine resources. The behavior of the Deep learning CNN model is studied and the performance has been measured. The obtained results are compared with the obtained test results from the Super-vector coding of local image descriptors method, SVM method, and Region Ranking SVM method, which tested with the PASCAL VOC 2007 data-set. The final results evaluate the deep learning algorithm as a state-of-the-art method for an image classification task.

Recognition of Human Emotion Detection and Annotation, using Local Descriptor and Support Vector Machine

In multimedia data analysis, video tagging is the most challenging and active research area. In which finding or detecting the object with the dynamic environment is most challenging. Object detection and its validation are an essential functional step in video annotation. Considering the above challenges, the proposed system designed to presents the people detection module from a complex background. Detected persons are validated for further annotation process. Using publically available dataset for module design, Viola-Jones object detection algorithm is used for person detection. Support Vector Machine (SVM) authenticate the detected object/person based on it local features using Local Binary Pattern (LBP). The performance of the proposed system presents given architecture is effectively annotating the detected people emotion.

Lychee Fruit Detection Based on Monocular Machine Vision in Orchard Environment.

Due to the change of illumination environment and overlapping conditions caused by the neighboring fruits and other background objects, the simple application of the traditional machine vision method limits the detection accuracy of lychee fruits in natural orchard environments. Therefore, this research presented a detection method based on monocular machine vision to detect lychee fruits growing in overlapped conditions. Specifically, a combination of contrast limited adaptive histogram equalization (CLAHE), red/blue chromatic mapping, Otsu thresholding and morphology operations were adopted to segment the foreground regions of the lychees. A stepwise method was proposed for extracting individual lychee fruit from the lychee foreground region. The first step in this process was based on the relative position relation of the Hough circle and an equivalent area circle (equal to the area of the potential lychee foreground region) and was designed to distinguish lychee fruits growing in isolated or overlapped states. Then, a process based on the three-point definite circle theorem was performed to extract individual lychee fruits from the foreground regions of overlapped lychee fruit clusters. Finally, to enhance the robustness of the detection method, a local binary pattern support vector machine (LBP-SVM) was adopted to filter out the false positive detections generated by background chaff interferences. The performance of the presented method was evaluated using 485 images captured in a natural lychee orchard in Conghua (Area), Guangzhou. The detection results showed that the recall rate was 86.66%, the precision rate was greater than 87% and the F1-score was 87.07%.

High-speed Railway Fastener Detection Using Minima Significant Region and Local Binary Patterns

Railway fastener is an important part of the railway system. Keeping the fasteners effective is essential to ensure the safe operation of the railway, so abnormal railway fastener detection is a main task of railway maintenance. With the development of railway system, the traditional manual fastener detection method has been unable to meet the application requirements, because it is very slow, costly, and dangerous. In this paper, we propose a novel method for abnormal fastener detection based on computer vision, which can detect missing fasteners automatically. In this method, Minima Significant Region is extracted in order to improve the fastener localization accuracy. Then, fastener recognition is operated using local binary features and Support Vector Machine classifier based on the fastener sub-images which are obtained by fastener localization. The proposed method is evaluated in our own database which is obtained by railway inspection system in different environments. The experimental results have shown improved performance against the state-of-the-art algorithm.

Automatic Learning Algorithms for Local Support Vector Machines

In this paper, we propose automatic learning algorithms (called auto-kSVM-1p, autokSVM-dp) being possible to automatically tune hyper-parameters of k local support vector machines (SVM) for classifying large data sets. The autokSVM algorithms are able to determine the number of clusters k to partition the large training data, followed which they auto-learn the non-linear SVM model in each cluster to classify the data locally in the parallel way on multi-core computers. The autokSVM algorithms combine the grid search, the .632 bootstrap estimator, the hill climbing heuristic to optimize hyper-parameters in the local non-linear SVM training. The numerical test results on four data sets from UCI repository and three benchmarks of handwritten letters recognition showed that our autokSVM algorithms give very competitive classification results compared to the standard LibSVM and the original kSVM. An example of autokSVM-1p’s effectiveness is given with an accuracy of 96.74% obtained in the classification of Forest cover-type data set having 581,012 datapoints in 54-dimensional input space and 7 classes in 334.45 seconds using a PC Intel(R) Core i7-4790 CPU, 3.6 GHz, 4 cores.