Support Vector Data Description Model Research Articles

Robust Support Vector Data Description with Truncated Loss Function for Outliers Depression.

Support vector data description (SVDD) is widely regarded as an effective technique for addressing anomaly detection problems. However, its performance can significantly deteriorate when the training data are affected by outliers or mislabeled observations. This study introduces a universal truncated loss function framework into the SVDD model to enhance its robustness and employs the fast alternating direction method of multipliers (ADMM) algorithm to solve various truncated loss functions. Moreover, the convergence of the fast ADMM algorithm is analyzed theoretically. Within this framework, we developed the truncated generalized ramp, truncated binary cross entropy, and truncated linear exponential loss functions for SVDD. We conducted extensive experiments on synthetic and real-world datasets to validate the effectiveness of these three SVDD models in handling data with different noise levels, demonstrating their superior robustness and generalization capabilities compared to other SVDD models.

Selecting Indispensable Edge Patterns With Adaptive Sampling and Double Local Analysis for Data Description

Support vector data description (SVDD) inspires us in data analysis, adversarial training, and machine unlearning. However, collecting support vectors requires pricey computation, while the alternative boundary selection with O(N2) is still a challenge. The authors propose an indispensable edge pattern selection method (IEPS) for data description with direct SVDD model building. IEPS suggests a double local analysis to select the global edge patterns. Edge patterns belong to a subset of the target problem of SVDD and its variants, and neighbor analysis becomes pivotal. While an excessive number of participating data result in redundant computations, an insufficient number may impede data separability or compromise the model's quality. Consequently, a data-adaptive sampling strategy has been devised to ascertain an optimal ratio of retained data for edge pattern selection. Extensive experiments indicate that IEPS keeps indispensable edge patterns for data description while reducing the interference in the norm vector generation to guarantee the effectiveness for clustering analysis.

Machinery Fault Signal Detection with Deep One-Class Classification

Fault detection of machinery systems is a fundamental prerequisite to implementing condition-based maintenance, which is the most eminent manufacturing equipment system management strategy. To build the fault detection model, one-class classification algorithms have been used, which construct the decision boundary only using normal class. For more accurate one-class classification, signal data have been used recently because the signal data directly reflect the condition of the machinery system. To analyze the machinery condition effectively with the signal data, features of signals should be extracted, and then, the one-class classifier is constructed with the features. However, features separately extracted from one-class classification might not be optimized for the fault detection tasks, and thus, it leads to unsatisfactory performance. To address this problem, deep one-class classification methods can be used because the neural network structures can generate the features specialized to fault detection tasks through the end-to-end learning manner. In this study, we conducted a comprehensive experimental study with various fault signal datasets. The experimental results demonstrated that the deep support vector data description model, which is one of the most prominent deep one-class classification methods, outperforms its competitors and traditional methods.

An unsupervised end-to-end approach to fault detection in delta 3D printers using deep support vector data description

Fault detection in 3D printers is crucial for safety and quality assurance, emphasizing proactive prediction over reactive rectification based on manufacturing factors. Presently, most detection techniques rely on shallow models with limited representational capabilities, necessitating manual feature extraction from the captured signals. This manual process is not only cumbersome and potentially costly but often requires intricate domain-specific knowledge. Additionally, these handcrafted features might not optimally distinguish between normal and faulty samples, potentially reducing prediction accuracy. In this study, we introduce an end-to-end approach using the Deep Support Vector Data Description model for fault detection in 3D printers. This design inherently facilitates automatic feature learning, where the features are synergistically optimized for fault detection. Our experiments leverage magnetic field signals for fault detection in 3D printers, using 1D convolutional layers to discern temporal signal patterns and wide kernels in the initial layer to mitigate high-frequency noise. Furthermore, our model can be easily adapted to integrate multi-channel signals for enhanced accuracy. Evaluations on real-world data from a delta 3D printer underscore the superiority of our method compared to existing alternatives.

A compound fault diagnosis model for photovoltaic array based on 1D VoVNet-SVDD by considering unknown faults

As photovoltaic (PV) arrays are exposed to the outdoors year-round, they are susceptible to various faults. The shading condition, degradation or dust coverage can make fault signals more complex, forming compound faults. These faults can lead to a large loss of power generation or irreversible damage to the PV modules, and even fires in severe cases. Moreover, unknown fault types that have never been seen in the training set may occur at actual working conditions. Therefore, accurate diagnosis of various types of single and compound faults (closed-set faults) by considering the identification of unknown faults, namely open-set faults diagnosis, is crucial to improve the efficiency of operation and maintenance. A 1D VoVNet-SVDD based open-set fault diagnosis model for PV arrays is proposed. The model is a two-stage network model consisting of a 1D VoVNet network and a multi-classification Support Vector Data Description (SVDD) in series. The 1D VoVNet network automatically extracts fault features from the input original I-V curve data. These extracted fault features are then combined with environmental parameters to construct the SVDD model. The SVDD identifies known fault types by constructing a hypersphere for each fault type. Fault types that are not classified into any of the hyperspheres are considered as unknown faults, enabling open-set diagnosis. The experimental results show that the proposed model can accurately classify the closed-set faults among the three designed testing tasks while identify unknown type faults. The comparison demonstrates that the proposed algorithm is superior to the compared models.

A novel quality inspection method of compressors based on Deep SVDD and CWGAN-XGBoost

To ensure the quality of manufactured compressors, the quality inspection should be performed before they leave the factory. The quality inspection process consists of two steps: quality assessment and fault diagnosis. However, the traditional fixed threshold range setting method used for quality assessment cannot fully consider the correlation among the operating parameters, and the test data of faulty compressors are often inadequate for fault diagnosis. To address these two issues, a novel quality inspection method based on Deep Support Vector Data Description (SVDD) and Conditional Wasserstein Generative Adversarial Nets (CWGAN)-Extreme Gradient Boosting (XGBoost) is presented. The Deep SVDD model is setup and trained for compressor quality assessment, while the CWGAN model is used to generate fake faulty samples, and the generated faulty samples are then used to train the XGBoost model for fault diagnosis. The presented method is validated by quality record data of two types of compressors, and the results show that the Deep SVDD model can significantly improve the accuracy of compressor quality assessment. Comparing with the fixed threshold range setting methods, the quality assessment accuracy for these two types of compressors increases by more than 3.77 % and 3.12 % respectively. The validation results also show that CWGAN model can generate suitable faulty samples for training XGBoost model, and the trained XGBoost model can accurately diagnose the manufacturing faults. The fault diagnosis accuracy for these two types of compressors is 81.43 % and 81.9 % respectively.

Constraint Guided Autoencoders to Enforce a Predefined Threshold on Anomaly Scores: An Application in Machine Condition Monitoring

Anomaly detection (AD) is an important task in a broad range of domains. A popular choice for AD are Deep Support Vector Data Description models. When learning such models, normal data is mapped close to and anomalous data is mapped far from a center, in some latent space, enabling the construction of a sphere to separate both types of data. Empirically it was observed: (i) that the center and radius of such sphere largely depends on the training data and model initialization which leads to difficulties when selecting a threshold, and (ii) that the center and radius of this sphere strongly impacts the model AD performance on unseen data. In this work, a more robust AD solution is proposed that (i) defines a sphere with a fixed radius and margin in some latent space and (ii) enforces the encoder, which maps the input to a latent space, to encode the normal data in a small sphere and the anomalous data outside a larger sphere, with the same center. Experimental results indicate that the proposed algorithm attains higher performance compared to alternatives and that the difference in size of the two spheres has a minor impact on the performance.

Sound Damage Detection of Bridge Expansion Joints Using a Support Vector Data Description.

A novel method is proposed for the damage identification of modal bridge expansion joints (MBEJs) based on sound signals. Two modal bridge expansion joint specimens were fabricated to simulate healthy and damaged states. A microphone was used to collect the impact signals from different specimens. The wavelet packet energy ratio of the sound signal was used to identify the difference in specimen state. Firstly, the wavelet packet energy ratio was used to establish the feature vectors, which were reduced dimensionality using principal component analysis. Subsequently, a support vector data description model was established to detect the difference in the signals. The identification effects of three parameter optimization methods (particle swarm optimization, genetic algorithm optimization, and Bayesian optimization) were compared. The results showed that the wavelet packet energy ratio of sound signals could effectively distinguish the state of the support bar. The support vector data description of Bayesian optimization worked best, and the proposed method could successfully detect damage to the support bar of MBEJs with an accuracy of 99%.

Robust one-class classification with support vector data description and mixed exponential loss function

Support vector data description (SVDD) has received a lot of attention due to its outstanding performance to perform one-class classification or novelty detection tasks. However, the same weight is directly imposed on all slack variables in the process of modeling, which may result in degraded learning performance when training data are contaminated by some outliers or mislabeled observations. In this paper, an extended SVDD model is therefore proposed by reformulating the original optimization problem of SVDD with a mixed exponential loss function. Since this loss function can emphasize the importance of the samples that tend to be the target class, and weaken the influence of those tending to be outliers, it can be viewed as a weighted SVDD. However, the weights in the new model are automatically calculated rather than being calculated in advance using some specific manners. To solve the optimization problem of the proposed model effectively, the half-quadratic optimization technique has been adopted to perform the optimization, generating a dynamic optimization algorithm. Meanwhile, the convergence and computational complexity of this dynamic optimization algorithm are analyzed from a theoretical respective. Experimental results on a synthetic data set and some publicly available real world data sets are reported to demonstrate the performance superiority of the new method in comparison with SVDD and other competitive SVDD variants.

Rolling Bearing Performance Degradation Assessment with Adaptive Sensitive Feature Selection and Multi-Strategy Optimized SVDD.

In light of the problems of a single vibration feature containing limited information on the degradation of rolling bearings, the redundant information in high-dimensional feature sets inaccurately reflecting the reliability of rolling bearings in service, and assessments of the degradation performance being disturbed by outliers and false fluctuations in the signal, this study proposes a method of assessing rolling bearings' performance in terms of degradation using adaptive sensitive feature selection and multi-strategy optimized support vector data description (SVDD). First, a high-dimensional feature set of vibration signals from rolling bearings was extracted. Second, a method combining the Technique for Order Preference by Similarity to an Ideal Solution (TOPSIS) and K-medoids was used to comprehensively evaluate the features with multiple evaluation indicators and to adaptively select better degradation features to construct the sensitive feature set. Next, multi-strategy optimization of the SVDD model was carried out by introducing the autocorrelation kernel regression (AAKR) and a multi-kernel function to improve the ability of the evaluation model to overcome outliers and false fluctuations. Through validation, it could be seen that the method in this study uses samples of rolling bearings in the healthy early stage to establish the evaluation model, which can adaptively determine the starting point of the bearing's degradation. The stability and accuracy of the model were effectively improved.

Unsupervised Anomaly Detection for Intermittent Sequences Based on Multi-Granularity Abnormal Pattern Mining

In the actual maintenance of manufacturing enterprises, abnormal changes in after-sale parts demand data often make the inventory strategies unreasonable. Due to the intermittent and small-scale characteristics of demand sequences, it is difficult to accurately identify the anomalies in such sequences using current anomaly detection algorithms. To solve this problem, this paper proposes an unsupervised anomaly detection method for intermittent time series. First, a new abnormal fluctuation similarity matrix is built by calculating the squared coefficient of variation and the maximum information coefficient from the macroscopic granularity. The abnormal fluctuation sequence can then be adaptively screened by using agglomerative hierarchical clustering. Second, the demand change feature and interval feature of the abnormal sequence are constructed and fed into the support vector data description model to perform hypersphere training. Then, the unsupervised abnormal point location detection is realized at the micro-granularity level from the abnormal sequence. Comparative experiments are carried out on the actual demand data of after-sale parts of two large manufacturing enterprises. The results show that, compared with the current representative anomaly detection methods, the proposed approach can effectively identify the abnormal fluctuation position in the intermittent sequence of small samples, and also obtain better detection results.

Online Anomaly Detection for Aeroengine Gas Path Based on Piecewise Linear Representation and Support Vector Data Description

The online anomaly detection of an aeroengine gas path system is very important to ensure the safety of aircraft. The traditional online anomaly detection method based on a sliding window has many shortcomings, which leads to the lack of accuracy of anomaly detection. In addition, the extreme lack of abnormal samples is also a difficult problem for engine anomaly detection. Therefore, this article proposes an anomaly detection method based on piecewise linear representation (PLR) and support vector data description (SVDD). First, the monitoring data of the aeroengine gas path are segmented and linearly represented so that the data in each segment have basically consistent changing trends, which can make up for the defects of the traditional sliding window. And an improved online segmentation method is proposed that increases the efficiency of the segmentation algorithm. Then, segment features are extracted, and input samples for the anomaly detection model are constructed. The classification model employs SVDD that requires only normal samples. The SVDD model is trained on the offline normal samples and anomaly detection is performed on the online samples. Experimental results show the superiority of this method in aeroengine gas path anomaly detection.

An Integrated Cluster Detection, Optimization, and Interpretation Approach for Financial Data.

In many financial applications, such as fraud detection, reject inference, and credit evaluation, detecting clusters automatically is critical because it helps to understand the subpatterns of the data that can be used to infer user's behaviors and identify potential risks. Due to the complexity of human behaviors and changing social environments, the distributions of financial data are usually complex and it is challenging to find clusters and give reasonable interpretations. The goal of this study is to develop an integrated approach to detect clusters in financial data, and optimize the scope of the clusters such that the clusters can be easily interpreted. Specifically, we first proposed a new cluster quality evaluation criterion, which is free from large-scale computation and can guide base clustering algorithms such as k -Means to detect hyperellipsoidal clusters adaptively. Then, we designed a new solver for a revised support vector data description model, which efficiently refines the centroids and scopes of the detected clusters to make the clusters tighter such that the data in the clusters share greater similarities, and thus, the clusters can be easily interpreted with eigenvectors. Using ten financial datasets, the experiments showed that the proposed algorithm can efficiently find reasonable number of clusters. The proposed approach is suitable for large-scale financial datasets whose features are meaningful, and also applicable to financial mining tasks, such as data distribution interpretation and anomaly detection.

Network Security Monitoring by Combining Semi-Supervised Learning and Active Learning

In network intrusion and network security monitoring, there is massive data. When using supervised learning method directly, it will cost lots of time to collect labeled samples, which is expensive. In order to solve this issue, this paper adopts an active learning model to detect network intrusion. First, massive unlabeled samples are used to establish a weighted support vector data description model. Then, the most valuable samples are used to improve the performance of network intrusion by combining with active learning, which utilizes labeled samples and unlabeled samples to extend the weighted support data description model in a semi-supervised learning method. The experimental results show that the active learning can utilize minor labeled sample to reduce the cost of manual labeling work, which is more suitable for an actual network intrusion detection environment.

High-Precision Fault Detection for Electric Vehicle Battery System Based on Bayesian Optimization SVDD

Fault detection of the electric vehicle battery system is vital for safe driving, energy economy, and lifetime extension. This paper proposes a data-driven method to achieve early and accurate battery system fault detection to realize rapid early warning. The method first adopts the support vector data description model mapping the feature of unlabeled voltage and temperature into a minimum volume hypersphere in high-dimensional space. When the feature is located outside the hypersphere, it is judged to be faulty. Then, to overcome the problem of hyperparameters selection, Bayesian optimization and a small amount of label data are used to iteratively train the model. This step can greatly improve the fault detection ability of the model, which is conducive to mining early and minor faults. Finally, the proposed model is compared with three unsupervised fault detection models, principal component analysis, kernel principal component analysis, and support vector data description to validate the performance of fault detection and robustness, respectively. The experimental results show that: 1. the proposed model has high detection accuracy in all four fault datasets, especially in the highly concealed cumulative short-circuit fault, which is substantially ahead of the other three models; and 2. The proposed model has higher and more stable accuracy than the other three models even in the case of a large range of signal-to-noise ratio.

A Three-Stage Low-Complexity Human Fall Detection Method Using IR-UWB Radar

This paper proposes a novel three-stage low-complexity human fall detection method using an impulse radio ultra-wideband (IR-UWB) radar. The core idea lies in the three cascaded stages, namely large-motion detection, rough fall detection and enhanced fall detection. For the large-motion detection, we assume the fall is a very sparse event in daily life and achieve this by a checking of the high Doppler frequency energy. For the rough fall detection, we do not intuitively determine the fall events, but propose six time-frequency features and two position features, and use the support vector data description (SVDD) detector to divide the large-motions into non-fall and fall-like events. For the enhanced fall detection, we add a new feature and use a Mahalanobis distance classifier to finally determine whether a fall happened. The reasons that two classifiers cascaded instead of one classifier are that (1) we can reduce the difficulty of identifying falls directly from daily events by using a large number of non-fall samples to train the SVDD model for anomaly detection, and allowing a certain false alarm rate; and (2) we can achieve a higher fall detection accuracy in a much smaller searching space by identifying falls only from the fall-like events. Additionally, a real-time edge fall detection system with a commonly used micro control unit is developed. Experiment results show that the proposed method exhibits a low computational complexity, and a relative robustness and high fall detection accuracy under a low false alarm rate.

A data-driven algorithm to detect false data injections targeting both frequency regulation and market operation in power systems

This paper focuses on detecting cyber-attacks targeting the Automatic Generation Control (AGC) loop and market operation. To achieve this, a new data-driven learning algorithm is proposed that ensembles various learning tools such as K-Means clustering, Synthetic Minority Oversampling Technique oversampling, and Support Vector Data Description models as the base learners. Next, this paper devises a new feature variable, EF, which generates a relatively higher value for attack scenarios than normal grid states, thus aiding the classifier in predictions with low false alarms. The proposed approach can detect a wide range of cyber-attacks, including unseen attack cases. This paper further modelled and validated the detection of profit-oriented intelligent market operation attacks, absent in most of the previous works. Such attacks project themselves within the proximity of nominal grid states, making them difficult to predict. The algorithm's performance is finally compared with other learning models, and it is found that the proposed technique has superior prediction with True Positive Rate, True Negative Rate, and Geometric Accuracy as 98.13%, 99.85%, and 98.98%, respectively.

Abnormal actions detection of robotic arm via 3D convolution neural network and support vector data description

Robotic arms are powerful assistants in many industrial production environments, and they run periodically in accordance with preset actions to complete specified operations. However, they may act abnormally when encountering unexpected situation and then lead to unnecessary loss. Recognizing the abnormal actions of robotic arms through surveillance video can automatically help us to understand their operating status and discover possible abnormalities in time. We designed a deep learning architecture based on 3D convolution for abnormal action recognition. The 3D convolutional layer can extract the spatial and temporal features of the robotic arm movements from the video frame difference sequence. The features are compressed and streamlined by the maximum pooling layer to obtain concise and effective robotic arm action features. Finally, the fully connected layer is used to classify the features to recognize the abnormal robotic arm tasks. Support vector data description (SVDD) model is employed to detect abnormal actions of the robotic arm, and the well-trained SVDD model can distinguish the normal actions from the three kinds of abnormal actions with the Area Under Curve (AUC) 99.17%.

Dynamic nonlinear process monitoring based on dynamic correlation variable selection and kernel principal component regression

In actual industrial processes, data are usually time series. Each process variable may have strong autocorrelation and cross-correlation with other variables with different delays. In addition, there are usually complex nonlinearities among variables. To further improve the monitoring performance for dynamic nonlinear processes, establishing a nonlinear filtering model for each variable is necessary. Therefore, a novel dynamic nonlinear process monitoring method based on dynamic nonlinear feature selection and kernel principal component regression (KPCR) is proposed in this study. First, dynamic nonlinear related variables are selected for each variable through mutual information by considering variables with different time delays. Second, process variables are divided into response and independent variable sets. Third, corresponding KPCR models are established to describe the dynamic relationships with the selected dynamic related variables as input variables and with response variables as output variables. To monitor the dynamic processes, kernel principal component analysis model is constructed on the basis of the residuals, where the residuals are obtained by comparing measured values of instruments with the predicted values of KPCR models. A support vector data description model is established to monitor the independent variables. Three cases are used to verify the performance of the novel approach. Results show that the proposed method is superior to and more effective than other advanced dynamic process monitoring methods.

Enhanced multiclass support vector data description model for fault diagnosis of gears

This work reports a study aimed at identifying the operating state of gear through an enhanced multiclass support vector data description (eMSVDD) model where the multiple hyperspheres corresponding to different operating states of gears are established. The model addresses the following issues: (1) the overlap of multiple hyperspheres deteriorates the diagnostic performance for boundary samples and severely weakens the generalization ability of the model; (2) the original support vector data description can only be applied to handle binary classification tasks; and (3) the performance of the SVDD depends heavily on the penalty factor C and kernel parameter δ. In the proposed model, we first implement the multi-label classification task by building multiple hyperspheres. Then, a novel fitness function for adaptive simplified chaotic particle swarm optimization is proposed to determine the model parameters, which considers the empirical risk minimization and model generalization capability. Finally, experimental results demonstrate that the eMSVDD outperforms deep neural networks and support vector machine by 3.4% and 0.5%, respectively, and this performance improvement respectively reaches 17.8% and 2.1% in the case of a very small sample size. The proposed approach provides a means for gear fault diagnosis with small samples.