Support Vector Domain Description Research Articles

Fuzzy regular least squares twin support vector machine and its application in fault diagnosis

Objective function ignores the generalization error in LSTSVM, and training overfitting results in poor generalization performance. All sample labels are considered deterministic, however, some samples contain outliers affected by noises, which leads to low reliability. A recognition method based on fuzzy regular LSTSVM (FRLSTSVM) is proposed. Firstly, L2 norm regular term is introduced into objective function to improve the generalization performance. Secondly, outliers of samples are detected through support vector domain description (SVDD), which improves outlier detection accuracy. Then, a membership degree S3 is constructed to give the outliers a suitable membership degree, reducing the impact of outliers on results. Finally, FRLSTSVM is extended to a multi-classification model by one versus one (OVO) and binary tree (BT) strategies, and it is combined with an improved multiscale fluctuating Rényi dispersion entropy (IMFRDE) for fault diagnosis. The results show that the method has stronger generalization, lower sensitivity to parameters and higher reliability.

A railway intrusion detection method based on decomposition and semi-supervised learning for accident protection

In recent years, video surveillance has become increasingly popular in railway intrusion detection. However, it is still quite challenging to detect the intruded objects efficiently and accurately because: (a) The backgrounds of video frames generated by the fixed cameras are similar and only few intrusive frames are available, resulting in a lack of diversity among video frames, and further leading to over fitting of the detection models during training; (b) The intrusion of small targets or targets far from the location of camera exhibits sparsity relative to the wide monitoring view of the camera, which challenges the detection of such targets in a complex background; (c) The extreme imbalance between non-intrusive frames and intrusive frames, as well as a large number of unlabeled frames, hinder the effective training of the detection model and weaken its capacity of generalization. To tackle the above issue, this article develops an effective intrusion detection method by combining low-rank and sparse decomposition (LRSD) and Semi-supervised Support Vector Domain Description (Semi-SVDD). Firstly, LRSD is used to decompose the monitored video into a background and a foreground. Then, based on the semantic segmentation method, we extract the mask of the track region in the decomposed background, which is used to mask the foreground. Next, by using both the labeled and unlabeled frames of the masked foreground, Semi-SVDD is established for the intrusion detection. Numerical results show that the removal of background interference and the combination of the labeled and unlabeled information help to improve the performance of the proposed method, and thus superior to benchmark methods.

Anomaly Detection Algorithm Based on Broad Learning System and Support Vector Domain Description

Deep neural network-based autoencoders can effectively extract high-level abstract features with outstanding generalization performance but suffer from sparsity of extracted features, insufficient robustness, greedy training of each layer, and a lack of global optimization. In this study, the broad learning system (BLS) is improved to obtain a new model for data reconstruction. Support Vector Domain Description (SVDD) is one of the best-known one-class-classification methods used to solve problems where the proportion of sample categories of data is extremely unbalanced. The SVDD is sensitive to penalty parameters C, which represents the trade-off between sphere volume and the number of target data outside the sphere. The training process only considers normal samples, which leads to a low recall rate and weak generalization performance. To address these issues, we propose a BLS-based weighted SVDD algorithm (BLSW_SVDD), which introduces reconstruction error weights and a small number of anomalous samples when training the SVDD model, thus improving the robustness of the model. To evaluate the performance of BLSW_SVDD model, comparison experiments were conducted on the UCI dataset, and the experimental results showed that in terms of accuracy and F1 values, the algorithm has better performance advantages than the traditional and improved SVDD algorithms.

Diabetic retinopathy screening using improved support vector domain description: a clinical study

Diabetic retinopathy screening using improved support vector domain description: a clinical study

A modified convolutional neural network architecture for diabetic retinopathy screening using SVDD

Automatic diabetic retinopathy diagnostic methods are proposed to facilitate the examination process and act as the physician’s helper. Most of the traditional convolution neural network (CNN) algorithms use only spatial features for image category recognition. This approach may not be optimal for the screening diabetic retinopathy because the retinal images have generally the same feature maps with minor differences in spatial domain. We propose a new high level image understanding using a modified CNN architecture mixed with modified support vector domain description (SVDD) as a classifier. This new innovative architecture uses two pathways extracting features of the retinal images in both spatial and spectral domains. The standard pre-trained AlexNet is chosen for modification to avoid the time complexity of the training algorithms. In spite the advantages of the modified AlexNet with two pathways configuration and standard SVDD classification, the different SVDD kernel functions affect the performance of the proposed algorithm. By using the appropriate transformed data into two or three dimensional feature spaces, the proposed SVDD can obtain more flexible and more accurate image descriptions. Also, we compared the performance of our approach with that of the commonly used as classification methods such as K-Means, subtractive and FCM clustering. Our proposed architecture achieves more than 98% precision and sensitivity for two class classification.

CAN Bus Messages Abnormal Detection Using Improved SVDD in Internet of Vehicles

Controller Area Network (CAN) bus anomaly detection on the Internet of Vehicle (IoV) is a topic of ongoing interest and increasing importance, particularly as IoV becomes commonplace. However, existing abnormal detection schemes are not ideal due to the lack of abnormal data in IoV and the difficulty of parsing the diverse message rules in existing vehicles. While single-classification of the SVDD (Support Vector Domain Description) algorithm can detect abnormalities only with normal message information, the approach has a high rate of false negatives when deployed directly in the in-vehicle network environment. In addition, the real-time data generation (e.g., IoV messages) compounds the challenge of designing effective detection approaches. Therefore, this paper proposes a mechanism for car networking message classification for a broad range of data, and establishes a weak model for many simple redundant data in the vehicle intranet. The detection can reduce the time and computation costs of detection while ensuring accuracy. Then, this paper proposes two improved SVDD schemes: M-SVDD scheme which adds the Markov chain to detect time-related messages; G-SVDD scheme which maps the kernel function to the Gaussian kernel function to reduce the redundant area of the model and the false-negative rate. The experimental results show that our proposed schemes have higher accuracy, recall rate and fewer computation overhead.

The scope of applicability of the selected class-modelling methods

Class-modelling methods are applied to construct a mathematical model based on the similarities among samples belonging to the same category, i.e., the target class. This model is used to study the belongingness of a new sample to the class for which the model was constructed. If the sample is recognised as not belonging to the target class, it is considered as an outlier. Therefore, the class-modelling techniques are widely used for food or drug authentication and confirmation of the product origin, in order to detect samples of poor quality or potential counterfeits. Structure of the target class data might suggest which of the available class-modelling approaches is well suited for model construction. Data structure can generally be described as normal, or heterogeneous. Normal structure exhibits Gaussian distribution, whereas heterogeneous data deviates from normal distribution, e.g., the objects can have multimodal distributions, create subgroups, and form complex shapes in the feature space. Class-modelling of the normally structured data can directly be performed on an original dataset, whereas heterogeneous datasets are usually subjected to kernel transformation prior to modelling. In this study, several datasets of various structures are analysed with different class-modelling methods to test their scope of applicability and the pros and cons from the practical point of view. It is revealed that in most cases, the Support Vector Domain Description (SVDD) leads to the best classification results. However, it is also discussed and illustrated that SVDD applied to a multimodal data can lead to sub-optimal model. In that case, the Potential Functions Method (PFM) is recommended for model construction. The PFM model is based on local densities of samples in the multivariate space and in that way, it tends to be more accurate than SVDD model. Additionally, some practical remarks are given on optimisation of the Gaussian kernel width.

A study of detection of abnormal network traffic: A comparison of multiple algorithms

AbstractThe detection of abnormal traffic in networks is of great value for maintaining network security. This article gives a brief introduction of abnormal traffic and compares the performance of three algorithms, the support vector domain description algorithm, the gradient boosting decision tree algorithm, and the extreme learning machine‐k‐nearest neighbor (ELM‐KNN) algorithm. Experiments were carried out on the NSL‐KDD dataset. It was found that the accuracies of the three algorithms were 0.8327, 0.8679, and 0.9468, the recall rates were 0.6764, 0.7236, and 0.8997, the F1‐scores were 0.7898, 0.8364, and 0.9578, and the false alarm rates were 0.3236, 0.2764, and 0.1003. The running time of the ELM‐KNN algorithm was far less than the other two algorithms. The experimental results verify the effectiveness of the ELM‐KNN algorithm in abnormal network traffic detection. The ELM‐KNN algorithm can be further promoted and applied in practice.

A novel two-stage hybrid default prediction model with k-means clustering and support vector domain description

Default prediction identifies the probability of a firm to default by establishing a prediction model. It reveals the functional relation between the features’ data at time t-m and default status at t. If the prediction of a defaulting company is wrong, it will mislead banks into making loans to a “defaulter,” causing huge losses; if the prediction of a non-defaulting company is wrong, it will result in a potential churn in high-quality customers. To support the lending decisions of banks and non-banking financial institutions, this study proposes a two-stage default prediction model that integrates k-means clustering for partitioning the sample and support vector domain description (SVDD) for predicting default (credit scoring). It also uses attributes’ data at time t-m (m = 1, 2, 3, 4, 5) and the default status at t to train the proposed model so that it can warn of default m years ahead. The results show that the predictive accuracy of the proposed two-stage default prediction model is better than that of single-stage models using only k-means clustering or support vector domain description, and the proposed model could achieve a five-year default prediction ability (AUC > 0.85). Further, the study implies that “retained earnings/total assets”, “financial expenses/gross revenue”, and “type of audit opinion” are three key features in default forecasting for Chinese listed enterprises. This study contributes to the field of multi-stage credit scoring research by demonstrating that a combination of different methods is worth considering to improve the performance of default prediction models.

Pseudo support vector domain description to train large-size and continuously growing datasets

Pseudo support vector domain description to train large-size and continuously growing datasets

The ensemble of density-sensitive SVDD classifier based on maximum soft margin for imbalanced datasets

Imbalanced problems have recently attracted much attention due to their prevalence in numerous domains of great importance to the data mining community. However, conventional bi-class classification approaches, e.g., Support vector machine (SVM), generally perform poorly on imbalanced datasets as they are originally designed to generalize from the training data, and pay little attention to the minority class. In the paper, we extend traditional support vector domain description (SVDD) and propose a novel density-sensitive SVDD classifier based on maximum soft margin (DSMSM-SVDD) for imbalanced datasets. In the proposed approach, the relative density-based penalty weights are incorporated into the optimization objective function to represent the importance of the data samples. Through optimizing the objective function with the relative density-based penalty weights, the training majority samples with high relative densities are more likely to lie inside the hypersphere, thus eliminating noise effects on traditional SVDD. In addition, to make full use of the minority class samples to refine the boundary in training, the maximum soft margin regularization term is also introduced in the proposed technique inspired by the idea of maximizing soft margin of traditional SVM. This method allows the optimal domain description boundary to more skew toward the minority class than traditional SVDD and thus improves the classification accuracy. Eventually, AdaBoost ensemble version of DSMSM-SVDD is developed so as to further improve the generalization performance and stability in dealing with imbalanced datasets. The extensive experimental results on various datasets demonstrate that the proposed approach significantly outperforms other existing algorithms when dealing with the imbalanced classification problems in terms of G-Mean, F-Measure and AUC performance measures.

A study of prime numbers distribution based on support vector domain description

Prime numbers are fundamental to the most common type of asymmetric encryption used today, namely Diffie-Hellman key exchange, Rivest-Shamir-Adleman cryptosystems, ElGamal Cryptosystem, Elliptic curve cryptosystems, etc. The strength of primes lies in the fact that they are the ‘building blocks’ of mathematics and in addition there is infinity of them with no apparent regularity in their distribution. Given these characteristics, they were used to build efficient one way functions used in public key cryptography. This work aims at exploring sets of prime numbers with the objective of finding patterns in their structure. For this purpose an efficient supervised learning model called Support Vector Domain Description was used as an explorer. The experimental results have shown a good generalization capability of this model to describe new prime numbers.

Authentication of honeybush and rooibos herbal teas based on their elemental composition

Rooibos (Aspalathus linearis) and honeybush (Cyclopia species) are popular indigenous herbal teas originating from South Africa. Both are enjoyed for their taste and aroma and more importantly, valued for their medicinal properties such as antioxidant, anti-diabetic, anti-inflammatory or immunomodulatory activity. In the European Union, rooibos and honeybush are protected as Geographical Indications. The Geographical Indication refers to products with unique characteristics that are related to their geographical origin. The authentication of products labelled as GIs is regarded as an issue of food quality and safety. Routine quality control procedures of GIs products prevent their fraud and counterfeiting on the market. However, techniques to determine adulteration or mislabelling of rooibos and honeybush do not exist yet. Therefore, in this study, the authentication of rooibos and honeybush based on their elemental composition was investigated. The methodology presented in this study combines energy-dispersive X-ray fluorescence spectrometry (EDXRF) for elemental analysis and a one-class classification approach. Elemental composition of plant material highly depends on soil on which the plant has been grown, but also on some other factors such as, e.g., atmospheric pollution, or plant metabolism. Determination of the elemental composition of the samples by EDXRF is non-destructive and does not require any complex sample preparation. One-class classification methods are well suited for authentication and origin verification problems. Based on the EDXRF data of samples from studied teas, individual class models were constructed for rooibos and honeybush. Several linear and nonlinear classification techniques were tested in order to find the model that handles the authentication task the best. For honeybush tea, the best classification results were obtained with the use of a nonlinear method based on Potential Functions. For rooibos tea, the highest classification outcomes were obtained by nonlinear One Class Partial Least Squares (OC-PLS) and the machine learning technique, Support Vector Domain Description (SVDD). The methodology implemented in the present study has the potential to be successfully applied for routine authentication of honeybush and rooibos teas.

An efficient method for modeling terrain and complex terrain boundaries in constrained wind farm layout optimization

A novel and efficient approach for modeling irregular domain boundaries and enforcing complex land-based constraints in wind farm layout optimization is presented. The presented method can be used to enforce constraints and represent complex domains for which an analytical expression is not available. The developed approach uses support vector domain description to obtain an analytical expression of the domain from data points sampled from the various regions for wind turbine placements. Such an analytical, continuous, differentiable description can then be used for wind farm layout optimization using gradient based and non-gradient based optimization algorithms. The applicability of the method was demonstrated on a wind farm layout optimization problem where several regions of the terrain were restricted. The method was shown to accurately and efficiently model regular, irregular, convex, non-convex and discontinuous domains/regions using only scattered data sampled from the domains. An efficient method for terrain modeling and interpolation is also presented. The proposed interpolation method uses quadtree decomposition to decompose the large interpolation problem into several smaller interpolation problems thereby reducing the complexity of the interpolation. The framework developed (libSVDD) is also made publicly available.

Multitask Learning Assisted Driver Identity Authentication and Driving Behavior Evaluation

The industrial Internet of Things has become the new driving force for the automobile industry, making people's travel increasingly convenient. However, there are still a multitude of challenges that need to be tackled, including but not limited to illegal driver detection, legal driver identification, and driving behavior evaluation. At present, many researchers have attempted to solve issues of illegal driver detection and legal driver identification by using deep learning network, but there are still quite a few limitations in the collection and analysis of driving behavior data. Moreover, the problem of driving behavior evaluation has been paid little attention. Therefore, in this article we conduct a comprehensive study on driving behavior habits and establish a multitask learning (MTL) network to solve the abovementioned problems. First, we collect original data from a real vehicle and extract the driving behavior characteristics. Then, a novel MTL network composed of long short-term memory network, support vector domain description model and feedforward neural network is established, which achieves illegal driver detection, legal driver identification, and driving behavior evaluation. Extensive experiments illustrate that the proposed MTL network not only supports parallel learning to reduce time and space costs, but also has excellent performances and robustness for the three tasks.

An SVDD-based post-processing approach for vibration risk assessment of the hydro-turbine-generator in a large hydropower station

Due to the frequently occurred adverse vibration of hydraulic structures, vibration risk assessment is significant for the water energy efficiency of hydropower station and the safety of people and structures. Recently, the abnormal vibration of hydro-turbine-generator in a large hydropower station occurred and the main influencing factors of vibration are analyzed based on the prototype data and engineering experience. Different from the deterministic variable features in traditional support vector domain description (SVDD) algorithms, the feature of vibration amplitude is actually a random variable so that the different target objects will be obtained at different confidence levels. In order to assess the vibration range and excessive vibration probability, the original SVDD boundary at relatively low confidence level is firstly calculated. Then, the boundary extension operation with detailed theoretical deduction is performed and the extended boundary is further optimized inspired by path planning problem. The advantage of proposed approach is that it can improve the data fitting performance for single dimension (i.e. vibration amplitude) without leading to complex boundary which cannot be used for vibration risk assessment. By applying this approach to the practical vibration problem, the quantitative and slightly conservative assessment results are conveniently obtained, which indicate that this approach is reasonable and cost-effective.

Fast and Memory-Efficient Import Vector Domain Description

One-class learning is a classical and hard computational intelligence task. In the literature, there are some effective and powerful solutions to address the problem. There are examples in the kernel machines realm, Support Vector Domain Description, and the recently proposed Import Vector Domain Description (IVDD), which directly delivers the sample probability of belonging to the class. Here, we propose and discuss two optimization techniques for IVDD to significantly improve the memory footprint and consequently to scale to datasets that are larger than the original formulation. We propose two strategies. First, we propose using random features to approximate the gaussian kernel together with a primal optimization algorithm. Second, we propose a Nystrom-like approximation of the functional together with a fast converging and accurate self-consistent algorithm. In particular, we replace the a posteriori sparsity of the original optimization method of IVDD by randomly selecting a priori landmark samples in the dataset. We find this second approximation to be superior. Compared to the original IVDD with the RBF kernel, it achieves high accuracy, is much faster, and grants huge memory savings.

Small Foreign Object Debris Detection for Millimeter-Wave Radar Based on Power Spectrum Features.

Foreign object debris (FOD) detection can be considered a kind of classification that distinguishes the measured signal as either containing FOD targets or only corresponding to ground clutter. In this paper, we propose a support vector domain description (SVDD) classifier with the particle swarm optimization (PSO) algorithm for FOD detection. The echo features of FOD and ground clutter received by the millimeter-wave radar are first extracted in the power spectrum domain as input eigenvectors of the classifier, followed with the parameters optimized by the PSO algorithm, and lastly, a PSO-SVDD classifier is established. However, since only ground clutter samples are utilized to train the SVDD classifier, overfitting inevitably occurs. Thus, a small number of samples with FOD are added in the training stage to further construct a PSO-NSVDD (NSVDD: SVDD with negative examples) classifier to achieve better classification performance. Experimental results based on measured data showed that the proposed methods could not only achieve a good detection performance but also significantly reduce the false alarm rate.

Hyperspectral multiple-change detection framework based on sparse representation and support vector data description algorithms

Change detection of hyperspectral (HS) images can be a motivating research area because of the dense spectral sampling of the HS dataset. The support vector domain description (SVDD) classifier has been widely used for change detection of multispectral images. Most of the research has focused only on the binary change detection problem. We took advantage of the high spectral resolution of HS images to extract different kinds of changes present in the image. As a result, we proposed a multiple-change detection algorithm for the HS dataset based on the sparse representation binary change mask and SVDD classifiers. Also, a geometric index, D2R2, was presented to integrate the binary SVDD classifiers. The experimental results showed that our proposed algorithm has comparable performance to some deep neural networks, including one-dimensional convolutional neural network (CNN), two-dimensional CNN, and Deep Pattern Net, with overall accuracies of 88.51% and 70.11% on the two Hyperion sensor datasets.

Using Feature Fusion Strategies in Continuous Authentication on Smartphones

With the increasing prevalence of mobile devices, people prefer to use smartphones to make payments, take photos, and collect personal vital information. Due to the high possibility of smartphone illegal access, the security and privacy of the devices become more important and critical. In this article, we present FusionAuth, a sensor-based continuous authentication system leveraging the accelerometer, gyroscope, and magnetometer on smartphones to capture users’ behavioral patterns. In order to improve the authentication performance and enhance system reliability, we are among the first to utilize two feature fusion strategies of serial feature fusion and parallel feature fusion to combine the designed features from the three sensors in the feature extraction module. Based on the trained one-class support vector domain description classifier, we evaluate the authentication performance of FusionAuth in terms of impact of window size and user size, and accuracy on different users. The experimental results demonstrate that FusionAuth reaches 1.47% mean balanced error rate with the serial fusion and achieves 1.79% mean BER with the parallel fusion.