Support Vector Data Description Algorithm Research Articles

The fuzzy support vector data description based on tightness for noisy label detection

AbstractMachine learning (ML) is an approach driven by data, and as research in machine learning progresses, the issue of noisy labels has garnered widespread attention. Noisy labels can significantly reduce the accuracy of supervised classification models, making it important to address this problem. Therefore, it is a very meaningful task to detect as many noisy labels as possible from the big data. In this study, a new method is proposed for detecting noisy labels in datasets. This method leverages a deep pre-trained network to extract a feature set from the image data first which can extract more accurate data features. Then, a membership degree based on tightness into the support vector data description (SVDD) model named TF-SVDD is introduced to detect noisy data in the dataset. In order to simulate different types of label noise more accurately, we first assumed that the labels of the datasets used were all correct, and in addition constructed the noise set using two method: the density peak noise set and the random noise set. Experimental results demonstrate that the TF-SVDD can effectively detect noisy label data, surpassing traditional support vector data description algorithms and other methods in terms of outlier detection accuracy, with the average accuracy mostly exceeding 50$$\%$$ % , and even reaching 80$$\%$$ % . Furthermore, one novel measure called ‘confidence’ is employed to rectify noisy labels in the data. Following the correction of noisy labels, the accuracy of image classification experiences a significant improvement, with the average promotion ratio mostly exceeding 10$$\%$$ % , and reaching 30$$\%$$ % .

Open‐set recognition of LPI radar signals based on a slightly convolutional neural network and support vector data description

AbstractLPI radar signal recognition based on convolutional neural networks usually assumes that the signal to be recognized belongs to a closed set of known signal classes. In an open electromagnetic signal environment, this type of closed‐set recognition method will experience a drastic drop in performance due to the encounter with unknown types of signals. We propose an SCNN‐SVDD model based on a combination of a lightweight convolutional neural network and a support vector data description algorithm to achieve open‐set recognition of LPI radar signals under unknown signal conditions. In this approach, Choi‐William's time‐frequency distribution is used to obtain two‐dimensional time‐frequency images of the signal to be identified, and convolutional neural networks are used to achieve high‐precision classification of known signals and extract the corresponding feature vectors. Then, the feature vectors are used as input to the SVDD algorithm and a hypersphere is constructed to detect whether the signal to be identified belongs to a known class. Experimental results show that the proposed method can detect unknown signals while maintaining high recognition accuracy for known signals.

Edge-on Low-surface-brightness Galaxy Candidates Detected from SDSS Images Using YOLO

Low-surface-brightness galaxies (LSBGs), fainter members of the galaxy population, are thought to be numerous. However, due to their low surface brightness, the search for a wide-area sample of LSBGs is difficult, which in turn limits our ability to fully understand the formation and evolution of galaxies as well as galaxy relationships. Edge-on LSBGs, due to their unique orientation, offer an excellent opportunity to study galaxy structure and galaxy components. In this work, we utilize the You Only Look Once object detection algorithm to construct an edge-on LSBG detection model by training on 281 edge-on LSBGs in Sloan Digital Sky Survey (SDSS) gri-band composite images. This model achieved a recall of 94.64% and a purity of 95.38% on the test set. We searched across 938,046 gri-band images from SDSS Data Release 16 and found 52,293 candidate LSBGs. To enhance the purity of the candidate LSBGs and reduce contamination, we employed the Deep Support Vector Data Description algorithm to identify anomalies within the candidate samples. Ultimately, we compiled a catalog containing 40,759 edge-on LSBG candidates. This sample has similar characteristics to the training data set, mainly composed of blue edge-on LSBG candidates. The catalog is available online at https://github.com/worldoutside/Edge-on_LSBG.

Power amplifier circuit defect detection based on improved Patch SVDD

AbstractIn the manufacturing process of power amplifier (PA), various defects on the circuit surface will seriously affect the circuit performance and its operation. To solve the above problems, this article proposes a circuit defect detection method based on improved patch‐based support vector data description algorithm (Patch SVDD), which uses contrastive learning to enhance the feature extraction ability of neural network. An image calibration algorithm is used for calibration preprocessing to enhance the robustness of the model. The Euclidean distance between a query image patch and its nearest normal patch is defined to be an anomaly score. For verification, three defects in the PA circuit, including components missing, solder contamination and component orientation dislocation, were detected. The experimental results show that, compared with conventional method, the area under receiver operating characteristic (AUROC) of the improved anomaly detection model increased from 90.1% to 95.1%, which improves the detection accuracy effectively.

Distribution Network Security Situation Awareness Method Based on the Distribution Network Topology Layered Model

Distribution network security situation awareness refers to the process of perception, understanding, and state projection of systems, elements, and environmental factors within a certain space-time volume. Security situation awareness is an important part of the security assessment of the distribution network. In response to the rapid and accurate distribution network security situation awareness requirements, this paper proposes a distribution network security situation awareness method based the distribution network topology layered model. First, a hierarchical model of the distribution network topology under the premise of optimizing the location of the synchronous phasor measuring device is constructed. This model can quickly capture the system security situation elements. Then, a support vector data description algorithm fused with information entropy is used to realize the identification and understanding of abnormal information in the security situation elements of the distribution network. The long- and short-term memory network is then used to predict the operation trend of the distribution network under normal operation and fault disturbance. Finally, a simulation is established, and the IEEE-33 distribution network model is used to verify the effectiveness of the method proposed in this paper. The results show that the method of this paper improves the speed and accuracy of obtaining the security situation elements of the distribution network, shortens the identification time of the security situation elements, and realizes the security situation awareness of the nodes of the distribution network.

A relaxed support vector data description algorithm based fault detection in distribution systems

The power detection of the distribution network is essential for reliable and secure distribution. In this paper, a flexible dual-threshold SVDD fault warning algorithm with fault samples is proposed to deal with problems concerning complex network topology, accessible data, and missing fault data in the power grid. For the problems of complicated network topology and a wide variety of signal types, we propose to combine wavelet packet energy features with Spearman to extract electric signal features, and finally achieve accurate feature extraction of multiple signal types. In the case of the problems of untimely judgement and low accuracy of the original SVDD, a relaxed SVDD fault warning algorithm with fault samples is correspondingly proposed. We turn the original SVDD boundary into a double-layer boundary, and divides the hypersphere space into three regions to increase the sensitivity to the fault samples and lessen the risk of missed detection. Besides, an adaptive update strategy is developed, which reduces the computational effort of the model and is proven more applicable to the distribution. Finally, the method is applied to numerical examples and fault detection experiments, and the experimental results in turn verified its effectiveness and superiority.

Foreign object detection for railway ballastless trackbeds: A semisupervised learning method

This paper proposes a semisupervised algorithm for detecting foreign objects in ballastless beds based on the improved deep SVDD (Support Vector Data Description) algorithm. First, we use the improved Mask R-CNN algorithm to extract the rail and fastener areas in images, assuming that no foreign object exists in the rail and fastener areas. Second, we deepen the backbone network of the deep SVDD to enhance its ability to extract deep semantics from complex images. We perform pure color coverage processing with different colors and mean blur processing with different blur kernels on the rails and fastener regions extracted by the improved Mask R-CNN. The results show that the AUC (Area Under the Curve) of our improved deep SVDD algorithm is 89.23% and improves the AUC compared to that of the original model by 11.09%.

Multi-Perspective Anomaly Detection.

Anomaly detection is a critical problem in the manufacturing industry. In many applications, images of objects to be analyzed are captured from multiple perspectives which can be exploited to improve the robustness of anomaly detection. In this work, we build upon the deep support vector data description algorithm and address multi-perspective anomaly detection using three different fusion techniques, i.e., early fusion, late fusion, and late fusion with multiple decoders. We employ different augmentation techniques with a denoising process to deal with scarce one-class data, which further improves the performance (ROC AUC ). Furthermore, we introduce the dices dataset, which consists of over 2000 grayscale images of falling dices from multiple perspectives, with 5% of the images containing rare anomalies (e.g., drill holes, sawing, or scratches). We evaluate our approach on the new dices dataset using images from two different perspectives and also benchmark on the standard MNIST dataset. Extensive experiments demonstrate that our proposed multi-perspective approach exceeds the state-of-the-art single-perspective anomaly detection on both the MNIST and dices datasets. To the best of our knowledge, this is the first work that focuses on addressing multi-perspective anomaly detection in images by jointly using different perspectives together with one single objective function for anomaly detection.

Improved multiclass support vector data description for planetary gearbox fault diagnosis

Planetary gearbox is one of the most important components of rotating machinery and plays a key role in modern industry. Due to the complex physical structures and harsh working conditions, planetary gearbox often suffers from different fault types, so it is of vital importance to investigate its fault diagnosis task. In this paper, a novel feature selection strategy is proposed to improve the multiclass support vector data description (SVDD) algorithm for planetary gearbox fault diagnosis. First, a novel feature selection method based on the cosine similarity measure in kernel space of Gaussian radial basis function (GRBF) is presented, so as to determine features that are sensitive to faults. Then, based on the selected features, an improved multiclass SVDD algorithm is developed to classify multiple classes of planetary gear faults, thus completing the fault diagnosis task. Finally, the effectiveness and advantage of the proposed method are demonstrated via experiments using wind turbine drivetrain diagnostics simulator (WTDDS), with comparison to several traditional methods.

An optimized railway fastener detection method based on modified Faster R-CNN

Accurate fastener positioning and state detection form the prerequisite for ensuring the safe operation of rail track. The demands for intelligent, fast and accurate detection cannot be satisfied by traditional methods using image processing and fastener classification. In view of this, a two-stage classification model based on the modified Faster Region-based Convolution Neural Network (Faster R-CNN) and the Support Vector Data Description (SVDD) algorithms is proposed in the paper for fastener detection. Firstly, the data set of detection images is built with the images being labeled, and the classification and detection model based on Faster R-CNN is constructed according to the characteristics of practical fastener images. The anchor box optimization function is established by labeled data set to optimize the box of region proposal network in the model, to enhance the detection rate and accuracy of detection. Then, according to the detection result by Faster R-CNN, the SVDD algorithm is applied for the second stage classification of deviated fasteners, which avoids inaccurate classification caused by different deviated angles of fasteners. Through the verification and analysis of practical detection case, it is verified that the proposed method can improve the efficiency and precision of fastener detection with higher detection rates and accuracy in comparison with other baseline detection methods, making it suitable for fast and accurate detection of fastener states.

Equipment Health Assessment Based on Improved Incremental Support Vector Data Description

With the rapid development of Internet-of-Things and big data, health assessment of equipment is receiving more attention in recent years. It is critical to bridge the gap between real-time production data and health status evaluation, which helps maintenance team understand the health status of equipment exactly, and then make rational maintenance plans. For this purpose, this paper proposes a framework to realize real-time equipment health assessment with health status quantitatively characterized by health degree (HD). The proposed framework begins with removing redundant features using a principal component analysis (PCA) method. Then, to represent the optimal operation status, a support vector data description (SVDD) algorithm is employed for extracting normal observations in the offline part. Thereafter, HD is introduced based on the Euclidean distance between current observation and the normal sample set. In order to achieve online updating of the normal sample set, and promote accuracy and computational efficiency of the offline part, an improved incremental SVDD algorithm based on adaptive threshold ${N}$ (NISVDD) is proposed. A case study is used to demonstrate the effectiveness of the proposed framework and model using a benchmark dataset of rolling bearing. Results suggest that the proposed framework is effective, and PCA shows good potential to extract features and keep most of the original information. The proposed NISVDD model is able to trace the dynamics of equipment health status for whole run-to-failure process, and outperforms other models in both accuracy and computational efficiency.

БИНАРНАЯ КЛАССИФИКАЦИЯ МНОГОАТРИБУТНЫХ РАЗМЕЧЕННЫХ АНОМАЛЬНЫХ СОБЫТИЙ КОМПЬЮТЕРНЫХ СИСТЕМ С ПОМОЩЬЮ АЛГОРИТМА SVDD

Introduction: At present, the volume of system logs of computer systems integrated into a distributed network infrastructure makes it impossible to manually check them in real time. Typically, the structure of each log record contains the numeric value of the observed attribute and a corresponding flag to mark the record as normal or abnormal. The support vector data description algorithm demonstrates high classification accuracy even with small volumes of the training sample. A feature of the algorithm is the work with a multi-attribute dataset, where each observation contains a common classifying marking. Consequently, the problem arises of reducing the set of markings of the attributes of the initial data to one marking of the entire observation. Purpose: to investigate the accuracy of the binary classification of experimental data of the Support Vector Data Description algorithm with a small volume of the training sample, provided that the data are labeled for each attribute separately. Methods: a method is proposed for solving the problem of reducing the set of markings of the attributes of the initial data to one single marking of the entire observation by means of two approaches: "normal observation" and voting by the majority principle. Two types of data are considered: ordered in time and uniformly mixed. The classification accuracy was assessed by calculating the area under the ROC curves with cross-validation for a different number of attributes. Results: a comparative analysis of observation labeling methods showed the advantage of the "completely normal observation" approach over the "majority vote" approach without "weighting". It is shown that the classification accuracy on mixed data is 7% higher compared to the variant of data ordering in time. The accuracy of the algorithm was investigated for a different number of attributes using the "completely normal observation" approach. The maximum achieved classification accuracy was about 96% when working with 6 attributes, with uniform mixing of the input dataset. A further increase in the number of attributes leads to a decrease in the average classification accuracy due to an increase in the proportion of anomalous observations. It is shown that when using uniform mixing of input data, the gain in accuracy can be increased by 15–20%. Practical relevance: the algorithm demonstrates an exponential growth in the consumption of computing resources with an increase in the amount of input data. Discussion: to achieve the maximum classification accuracy with acceptable resource consumption, it is necessary to form a compact set of input data, which most fully reflects the functioning of the computer system in normal mode.

Preload state detection for precision spindle bearings based on multi-level classification

In this study, a multi-level state classification method based on support vector data description (SVDD) is proposed to detect bearing preload state. Firstly, a three-category classification support vector data description algorithm is proposed to establish the three-state non-aliasing hypersphere model, which can combine the kernel principal component analysis (KPCA) and membership degree. Then, by classifying the first-level training sample model of uniform and non-uniform preload states, the second-level training sample model of non-uniform preload states is established based on SVDD algorithm. Moreover, the balanced multi-label propagation classification criterion is defined that can be used to identify the preload state level based on the training sample model. Finally, a preload state detection system is developed, which can accurately simulate uniform/non-uniform preload states for spindle bearings. The experimental results demonstrate that the proposed algorithm can effectively classify the preload states of spindle bearings with average accuracy higher than 94%.

Research on Kernel-Distance-Based AEWMA-t Control Method

Aimed at the complex data correlation among multivariate quality characteristics of products, inability to obey the assumptions of traditional control methods, low data volume in small and medium batch production, and inaccurate parameter estimation in process control, an AEWMA-t control method based on kernel distance data is proposed. Firstly, based on the support vector data description algorithm, the hypersphere is trained by normal samples. Then the kernel distance from the sample to the center of the hypersphere is calculated and the kernel distance is normally converted. Finally, the process is controlled with AEWMA-t control method. Case analysis shows that, compared with the traditional multivariate control methods, this method has good ability to detect anomalies in the mean deviation interval of each process, and is not restricted by the distribution and size of the data samples.

Hyperspectral multiple-change detection framework based on sparse representation and support vector data description algorithms

Change detection of hyperspectral (HS) images can be a motivating research area because of the dense spectral sampling of the HS dataset. The support vector domain description (SVDD) classifier has been widely used for change detection of multispectral images. Most of the research has focused only on the binary change detection problem. We took advantage of the high spectral resolution of HS images to extract different kinds of changes present in the image. As a result, we proposed a multiple-change detection algorithm for the HS dataset based on the sparse representation binary change mask and SVDD classifiers. Also, a geometric index, D2R2, was presented to integrate the binary SVDD classifiers. The experimental results showed that our proposed algorithm has comparable performance to some deep neural networks, including one-dimensional convolutional neural network (CNN), two-dimensional CNN, and Deep Pattern Net, with overall accuracies of 88.51% and 70.11% on the two Hyperion sensor datasets.

Unsupervised Anomaly Detection With LSTM Neural Networks.

We investigate anomaly detection in an unsupervised framework and introduce long short-term memory (LSTM) neural network-based algorithms. In particular, given variable length data sequences, we first pass these sequences through our LSTM-based structure and obtain fixed-length sequences. We then find a decision function for our anomaly detectors based on the one-class support vector machines (OC-SVMs) and support vector data description (SVDD) algorithms. As the first time in the literature, we jointly train and optimize the parameters of the LSTM architecture and the OC-SVM (or SVDD) algorithm using highly effective gradient and quadratic programming-based training methods. To apply the gradient-based training method, we modify the original objective criteria of the OC-SVM and SVDD algorithms, where we prove the convergence of the modified objective criteria to the original criteria. We also provide extensions of our unsupervised formulation to the semisupervised and fully supervised frameworks. Thus, we obtain anomaly detection algorithms that can process variable length data sequences while providing high performance, especially for time series data. Our approach is generic so that we also apply this approach to the gated recurrent unit (GRU) architecture by directly replacing our LSTM-based structure with the GRU-based structure. In our experiments, we illustrate significant performance gains achieved by our algorithms with respect to the conventional methods.

Railway vehicle bearings risk monitoring based on normal region estimation for no-fault data situations

A risk monitoring method based on normal region estimation (NRE) is systematically proposed for the actual situation of the lack of fault data in the condition identification and monitoring of railway vehicle bearings. First, the basic concept of normal domain theory is expounded, and the formal expression of normal domain is given. Secondly, the academic thoughts and implementation steps of risk monitoring based on NRE are summarized. Then, two algorithms based on convex hull and support vector data description (SVDD) are proposed respectively to solve the core problem of boundary estimation. Finally, the rolling-bearing vibration acceleration data was used for the experiment, and the performance of the two algorithms is compared. The results show that both algorithms are effective. In contrast, the convex hull algorithm is faster, and the SVDD algorithm is smoother and more flexible. In practical applications, the two algorithms can be selected according to different requirements of real time and accuracy.

Fault Diagnosis for Rolling Bearing Based on Semi-Supervised Clustering and Support Vector Data Description with Adaptive Parameter Optimization and Improved Decision Strategy

Rolling bearing is of great importance in modern industrial products, the failure of which may result in accidents and economic losses. Therefore, fault diagnosis of rolling bearing is significant and necessary and can enhance the reliability and efficiency of mechanical systems. Therefore, a novel fault diagnosis method for rolling bearing based on semi-supervised clustering and support vector data description (SVDD) with adaptive parameter optimization and improved decision strategy is proposed in this study. First, variational mode decomposition (VMD) was applied to decompose the vibration signals into sets of intrinsic mode functions (IMFs), where the decomposing mode number K was determined by the central frequency observation method. Next, fuzzy entropy (FuzzyEn) values of all IMFs were calculated to construct the feature vectors of different types of faults. Later, training samples were clustered with semi-supervised fuzzy C-means clustering (SSFCM) for fully exploiting the information inside samples, whereupon a small number of labeled samples were able to provide sufficient data distribution information for subsequent SVDD algorithms and improve its recognition ability. Afterwards, SVDD with improved decision strategy (ID-SVDD) that combined with k-nearest neighbor was proposed to establish diagnostic model. Simultaneously, the optimal parameters C and σ for ID-SVDD were searched by the newly proposed sine cosine algorithm improved with adaptive updating strategy (ASCA). Finally, the proposed diagnosis method was applied for engineering application as well as contrastive analysis. The obtained results reveal that the proposed method exhibits the best performance in all evaluation metrics and has advantages over other comparison methods in both precision and stability.

Monitoring Wind Turbines' Unhealthy Status: A Data-Driven Approach

Condition monitoring plays an important role in wind turbine maintenance, wherein monitoring turbines’ unhealthy statuses, mainly those not-runnable statuses causing turbines stop working, could be beneficial in both maintenance cost and wind power generation. A data-driven approach based on support vector data description (SVDD) and extreme learning machine (ELM) algorithms is proposed in this paper to realize effective monitoring on wind turbines’ unhealthy status. The SVDD algorithm is applied to separate data of unhealthy statuses from the healthy one. The ELM algorithm is used to construct a classifier for monitoring unhealthy (not-runnable) statuses of wind turbines. Industrial data from real wind farms are studied. Numerical results illustrate the feasibility of the proposed approach, and comparison studies with six monitoring algorithms validate that the proposed two-phase model could obtain better performance than other models.

Nonlinear Process Monitoring Based on Global Preserving Unsupervised Kernel Extreme Learning Machine

Recently, the unsupervised extreme learning machine (UELM) technique as a nonlinear data mining approach has been employed to diagnose nonlinear process faults. However, during the dimensionality reduction of process observation data, UELM only aims at mining the local structure feature information of process data and lacks of the ability of preserving the global structure analysis of process data, which would perform unsatisfactorily for monitoring nonlinear process. Furthermore, the choice of the optimal number of hidden layer nodes in UELM method is still a challenging problem. To handle these two tough problems, a new fault detection approach on the basis of global preserving unsupervised kernel extreme learning machine (GUKELM) technique is proposed to monitor nonlinear process effectively. In GUKELM technique, the data global structure preserving framework is naturally incorporated into standard UELM to capture the local structure information as well as to preserve the global structure feature information of nonlinear process observation data during the dimensionality reduction. Meanwhile, to tackle the strong nonlinearity of process observation data, the kernel trick is utilized to successfully solve the challenging issue of setting the optimal number of hidden layer nodes. Based on extracted process data low dimensional feature information from GUKELM method, support vector data description algorithm is adopted to construct the monitoring statistic to detect process fault. Finally, the feasibility and effectiveness of the proposed process monitoring strategy is illustrated through a numerical nonlinear system and the continuous stirred tank reactor process which is a typical nonlinear process.