Support Vector Data Description Method Research Articles

Fault detection in the marine engine using a support vector data description method

Fast detection and correct diagnosis of any engine condition changes are essential elements of safety and environmental protection. Many diagnostic algorithms significantly improve the detection of malfunctions. Studies on diagnostic methods are rarely reported and even less implemented in the marine engine industry. To fill this gap, this paper presents the Support Vector Data Description (SVDD) method as applied to the fault detection of the fuel delivery system of a two-stroke marine engine. The selected diagnostic data is the exhaust gas composition, with four components considered: oxygen, carbon oxide, nitric oxide, and carbon dioxide. With these diagnostics, the method distinguishes eight different engine faults from the efficient state. The manuscript presents in detail the methodology for applying the SVDD method in a marine engine. The method of obtaining diagnostic data and its scaling is described. The method of training and validating the algorithm is also presented, along with ready-made algorithms for use. The 100% accuracy of the proposed fault detection method. Based on the obtained results, the proposed fault detection method is promising for a simple application. Moreover, generalised algorithms that may be adapted to different technical solutions are also presented. Highlights SVDD was used for marine engine fault detection from exhaust gas composition Laboratory measurements were carried out on the two-stroke diesel engine The proposed algorithm detected the considered faults with 100% accuracy A generalised algorithm for adapting other complex technical objects was proposed

Large-margin multiple kernel [formula omitted]-SVDD using Frank–Wolfe algorithm for novelty detection

Using a variable ℓp≥1-norm penalty on the slacks, the recently introduced ℓp-norm Support Vector Data Description (ℓp-SVDD) method has improved the performance in novelty detection over the baseline approach, sometimes remarkably. This work extends this modelling formalism in multiple aspects. First, a large-margin extension of the ℓp-SVDD method is formulated to enhance generalisation capability by maximising the margin between the positive and negative samples. Second, based on the Frank–Wolfe algorithm, an efficient yet effective method with predictable accuracy is presented to optimise the convex objective function in the proposed method. Finally, it is illustrated that the proposed approach can effectively benefit from a multiple kernel learning scheme to achieve state-of-the-art performance.The proposed method is theoretically analysed using Rademacher complexities to link its classification error probability to the margin and experimentally evaluated on several datasets to demonstrate its merits against existing methods.

Patch SVDD(support vector data description)-based channel attention embedding and improvement of classifier

Some anomaly detection methods are based on CNN to fuse spatial and channel-wise information together within local receptive fields. However, the correlation between feature channels has not been fully utilized. Channel attention has been shown to model the interdependence between convolution feature channels and improve network representation. It is possible to introduce channel attention into anomaly detection. We attempt to directly embed the SE(Squeeze and Excitation) module into the convolutional layer but reduced anomaly detection performance. Therefore, we propose a lightweight channel attention module C-SE(Current Squeeze and Excitation) suitable for anomaly detection. C-SE module not only improves the representation ability of depth convolutional neural network but also has a significant effect on texture anomaly detection. C-SE module body is constructed by average pooling and maximum pooling branches, which ensure that local salient features of the image are not lost. Then reduce the negative impact of feature calibration through a long connection. In addition, the improvement of classifier plays an important role. Experimental results have shown that the proposed method outperforms the Patch SVDD methods by 3% in image-level AUROC and 0.7% in pixel-level AUROC on the MVTec benchmark. The higher AUROC score and the faster rate of convergence prove the effectiveness of the method.

A quality spectra-based SVDD method for multi-dimensional quality fluctuation evaluation in complex industrial process

To address the problem that fluctuation caused by multi-dimensional quality-related process variables is difficult to evaluate, this paper proposes a quality spectra-based support vector data description (SVDD) method for multi-dimensional quality fluctuation evaluation in complex industrial process. Firstly, based on quality spectra modelling, a multi-dimensional state feature vector of quality fluctuation is constructed from the perspective of quality spectra and statistical distribution, which is used to characterize the difference between normal and abnormal fluctuation. Then the quality fluctuation evaluation model is established based on SVDD algorithm. On this basis, the spherical distance between the sample to be evaluated and the quality evaluation model is calculated to measure the quality fluctuation degree. Finally, a case for TE process data sets is used to verify the quality fluctuation evaluation approach, it reveals that the proposed method could evaluate the degree of quality fluctuation, which could provide some theoretical guides for quality stability control.

An Object-Oriented Deep Multi-Sphere Support Vector Data Description Method for Impervious Surfaces Extraction Based on Multi-Sourced Data

The effective extraction of impervious surfaces is critical to monitor their expansion and ensure the sustainable development of cities. Open geographic data can provide a large number of training samples for machine learning methods based on remote-sensed images to extract impervious surfaces due to their advantages of low acquisition cost and large coverage. However, training samples generated from open geographic data suffer from severe sample imbalance. Although one-class methods can effectively extract an impervious surface based on imbalanced samples, most of the current one-class methods ignore the fact that an impervious surface comprises varied geographic objects, such as roads and buildings. Therefore, this paper proposes an object-oriented deep multi-sphere support vector data description (OODMSVDD) method, which takes into account the diversity of impervious surfaces and incorporates a variety of open geographic data involving OpenStreetMap (OSM), Points of Interest (POIs), and trajectory GPS points to automatically generate massive samples for model learning, thereby improving the extraction of impervious surfaces with varied types. The feasibility of the proposed method is experimentally verified with an overall accuracy of 87.43%, and its superior impervious surface classification performance is shown via comparative experiments. This provides a new, accurate, and more suitable extraction method for complex impervious surfaces.

Multiclass Support Vector Data Description in Extreme Learning Machine Feature Space

Support vector data description (SVDD) method aims to address the one-class classification (OCC) problem to find a hypersphere-shaped description of target data set. For extending SVDD to multiclass classification while remaining the ability of detecting outliers, we propose a novel multiclass SVDD scheme which can be used in effective feature mapping and meta-class separation based on the extreme learning machine algorithm (ELM-MSVDD). Accordingly, the imprecise data difficult to distinguish in specific classes is classified to a meta-class,the meta-class is defined by the disjunction of these specific classes, this operation can reduce the error rate effectively. Experimental results of our ELM-MSVDD method show well performance on the datasets from the UCI machine learning library and radar signal source recognition. Meanwhile, our proposed method provide a theoretical and practical support for other relevant pattern recognition field.

Fault diagnosis for rotor based on multi-sensor information and progressive strategies

Fault diagnosis is an effective tool to ensure safe operation of machinery and avoid serious accidents. As most currently used fault diagnosis methods usually employ mapping relationship established by training samples and their labels to achieve classification of testing samples, it is difficult for them to achieve fault diagnosis under the condition of incomplete training sample types. In addition, previous studies usually focus on feature extraction of single-channel vibration signal, which cannot get complete fault feature information. To solve the above problems, a progressive fault diagnosis method is investigated in this paper. First, the preliminary fault detection for the rotor is performed by studying reconstruction error of a sparse auto-encoder. Second, if a fault exists in the rotor, the outlier detection is implemented by the support vector data description method. Finally, if there are no outlier samples, the well -trained support vector machine is used to confirm the type of fault samples and complete the diagnosis. The performance of the proposed method was verified using the data obtained from a rotor laboratory bench. The types of rotor states investigated include normal, contact-rubbing, unbalance and misalignment. The experimental results verify the effectiveness and superiority of the proposed method in reducing the incidents of fault omission and fault misunderstanding.

Modified t-Distribution Stochastic Neighbor Embedding Using Augmented Kernel Mahalanobis-Distance for Dynamic Multimode Chemical Process Monitoring

The traditional data-driven process monitoring methods may not be applicable for the system which has dynamic and multimode characteristics. In this paper, a novel scheme named modified t-distribution stochastic neighbor embedding using augmented Mahalanobis-distance for dynamic multimode chemical process monitoring (AKMD-t-SNE) is proposed to realize dynamic multimodal process monitoring. First, the augmented matrix strategy is utilized to ensure the sample contains the autocorrelation of the process. Moreover, AKMD-t-SNE method eliminates the scale and spatial distribution differences among multiple modes by calculating the kernel Mahalanobis distance between the samples to establish the global model. The features extracted via the proposed method are obviously non-Gaussian, and there will be a deviation in the construction of traditional statistics. Then, the support vector data description (SVDD) method is used to construct statistics to deal with this problem. In addition, a hybrid correlation coefficient method (HCC) is proposed to achieve fault isolation and improve the accuracy of isolation results. The advantages of the proposed scheme are illustrated by a numerical case and the Multimode Tennessee Eastman Process (MTEP) benchmark.

Anomaly detection method for vehicular network based on collaborative deep support vector data description

With the accelerating convergence of fifth-generation (5G) communication technology and the automotive industry, vehicular network has received wide attention due to its great potential for improving safety and efficiency. However, it is still a great challenge to protect the driving safety and privacy of vehicular network from the increasingly complex network environment. Aiming at the problem of low detection accuracy in traditional vehicular network intrusion detection system (VIDS), a collaborative deep support vector data description (Coll-DSVDD) method combining multi-agent collaborative network and deep support data description (DSVDD) is proposed for optimization. The method firstly deploys the deep convolutional neural network and support vector data description in each vehicular network, and then performs collaborative interaction among multiple vehicular networks to generate an optimized network. The optimized network generated after coordination does not need to be retrained, which can effectively improve the protection capability of the vehicular network against new attacks while reducing the false negative rate (FNR) during detection. The experimental results show that, compared with the traditional single vehicular network detection system, the Coll-DSVDD method can optimize the FNR to 52% with an average detection accuracy of 91%, when it is deployed in multiple vehicular networks.

Multi-Scale Process Monitoring Based on Time-Frequency Analysis and Feature Fusion

Data-driven process monitoring is an important tool to ensure safe production and smooth operation. Generally, implicit information can be mined through data processing and analysis algorithms to detect process disturbances on the basis of historical production data. In industrial practice, signals with different sources of disturbance show different distribution patterns along with the time domain and frequency domain, that is, noise and pulse-type changes are usually contained in the high-frequency portion while most process dynamic is contained in the low-frequency portion. However, feature extraction is usually implemented at a single scale in traditional multivariate statistical algorithms. With this concern, a novel multi-scale process monitoring method is proposed in this work, by which wavelet packet decomposition is first employed for time-frequency analysis. After decomposition, multivariate statistical models are established for each scale to construct process statistics. For the high-frequency part, the classical principal component analysis (PCA) algorithm is adopted to construct squared prediction error (SPE) and Hotelling T2(T2) statistics. While for the low-frequency part, the slow feature analysis (SFA) algorithm is adopted to construct T2, Te2, S2 and Se2 statistics for the extraction of the long-term slowly changing trend. Then the monitoring statistics, obtained from each method at different scales, are integrated by a support vector data description (SVDD) method to give a final fault detection decision. The performance of the proposed method is verified on the benchmark Tennessee Eastman Process (TEP) and an industrial continuous catalytic reforming heat exchange unit by comparing with related multivariate statistical methods, which only focus on a single scale.

A novel adaptive weighted kernel extreme learning machine algorithm and its application in wind turbine blade icing fault detection

The conventional weighted kernel extreme learning machine (WKELM) is a typical cost-sensitive learning algorithm, which can provide a solution for the data imbalance problem in wind turbine blade icing fault detection. However, WKELM algorithm uses a fixed weighting strategy that only considers the number of samples to achieve imbalanced learning, which leads to its insufficient performance in data imbalance classification and fault detection. In this study, the support vector data description (SVDD) method is used to obtain sample distribution information to construct a novel adaptive weighting strategy. The adaptive weighting strategy is combined with the conventional fixed weighting strategy to construct an adaptive weighted kernel extreme learning machine (AWKELM) algorithm. The effectiveness and superiority of the AWKELM algorithm is verified through three benchmark cases, it is demonstrated that AWKELM algorithm can effectively solve the problem of data imbalance and further improve the effect of wind turbine blade icing detection.

Multicondition operation fault detection for chillers based on global density-weighted support vector data description

Fault detection is essential to maintain the healthy and efficient operation of chillers. However, in actual chiller operation, due to fluctuating operating conditions, fault detection becomes very difficult, i.e., a low detection accuracy and high false alarm rate (FAR) are attained. To better reflect the distribution of data, effectively improve the detection accuracy and reduce the FAR, this study considers the global perspective of target data, and a method based on global density-weighted support vector data description (GDW-SVDD) is proposed to detect chiller faults. This method is validated against ASHRAE RP-1043 experimental data. The results show that compared to the conventional SVDD method, the GDW-SVDD method not only effectively reduces the FAR from 10.75% to 8.25% but also improves the fault detection accuracy by 3.75% (under the reduced evaporator water flow fault at severity level 2). Compared to the density-weighted support vector data description (DW-SVDD) method, this method can effectively improve the accuracy of fault detection up to 6.5% (under the excessive oil fault at severity level 2) while simultaneously maintaining the same low FAR. Therefore, the proposed method is effective for chiller fault detection.

A Deep Support Vector Data Description Method for Anomaly Detection in Helicopters

Helicopters are high-value mechanical assets which has gained much attention from condition monitoring practitioners. Modern helicopter health management system leverages various sensors to collect in-flight signals. In order to trigger the alarm when an anomaly happens, signal processing methods are used to construct health indicators that require expert knowledge. On the other hand, classic features are always case-specific and may fail to discriminate anomalous in practical applications. Support Vector Data Description (SVDD) is a machine learning method used as a one-class classifier to serve anomaly detection tasks. It utilizes healthy samples to construct a hyper-sphere feature space as a detection threshold. In order to automate the anomaly detection pipeline, a deep SVDD model is proposed in this paper. A Convolution Neural Network (CNN) is used as the feature extractor, which provides smart features to an SVDD model. The SVDD model uses a soft-boundary hyper-sphere for decision-making. The optimization of the CNN and the SVDD is connected, which makes it an end-to-end process. The methodology is applied, tested and evaluated on a helicopter vibration dataset, which has been provided by Airbus SAS in the frames of AI Gym challenge. The experimental results reveal that the F1 score of the proposed Deep SVDD can reach 94%, showing its compelling efficacy for anomaly detection.

Deep Convolutional Feature-Based Probabilistic SVDD Method for Monitoring Incipient Faults of Batch Process

Support vector data description (SVDD) has been widely applied to batch process fault detection. However, it often performs poorly, especially when incipient faults occur, because it only considers the shallow data feature and omits the probabilistic information of features. In order to provide better monitoring performance on incipient faults in batch processes, an improved SVDD method, called deep probabilistic SVDD (DPSVDD), is proposed in this work by integrating the convolutional autoencoder and the probability-related monitoring indices. For mining the hidden data features effectively, a deep convolutional features extraction network is designed by a convolutional autoencoder, where the encoder outputs and the reconstruction errors are used as the monitor features. Furthermore, the probability distribution changes of these features are evaluated by the Kullback-Leibler (KL) divergence so that the probability-related monitoring indices are developed for indicating the process status. The applications to the benchmark penicillin fermentation process demonstrate that the proposed method has a better monitoring performance on the incipient faults in comparison to the traditional SVDD methods.

Anomaly detection using improved deep SVDD model with data structure preservation

• Typical SVDD structures are investigated in view of model depth profiles. • An improved deep SVDD model is proposed by considering the data structure preservation. • The experiments on the benchmark datasets verify the effectiveness of the proposed method. Support vector data description (SVDD) is a classical anomaly detection algorithm. How to develop a deep version of SVDD is one valuable problem in the anomaly detection field. Aiming at this problem, an improved SVDD model called deep structure preservation SVDD (DSPSVDD) is proposed by integrating the deep feature extraction with the data structure preservation. Firstly, the typical SVDD methods are revisited in view of model depth profiles and the limitations of the present deep SVDD model are analyzed. Then in order to extract the deep data features more effectively, an enhanced comprehensive optimization objective is designed for the deep SVDD model by considering both the hypersphere volume minimization and the network reconstruction error minimization simultaneously. The experimental results on the MNIST, Fashion-MNIST, and MVTec AD image benchmark datasets show that the proposed DSPSVDD method achieves the better anomaly detection performance compared with the traditional deep SVDD method.

Meta Learning for Few-Shot One-Class Classification

We propose a method that can perform one-class classification given only a small number of examples from the target class and none from the others. We formulate the learning of meaningful features for one-class classification as a meta-learning problem in which the meta-training stage repeatedly simulates one-class classification, using the classification loss of the chosen algorithm to learn a feature representation. To learn these representations, we require only multiclass data from similar tasks. We show how the Support Vector Data Description method can be used with our method, and also propose a simpler variant based on Prototypical Networks that obtains comparable performance, indicating that learning feature representations directly from data may be more important than which one-class algorithm we choose. We validate our approach by adapting few-shot classification datasets to the few-shot one-class classification scenario, obtaining similar results to the state-of-the-art of traditional one-class classification, and that improves upon that of one-class classification baselines employed in the few-shot setting.

A method of non-bug report identification from bug report repository

One of the most common issues addressed by bug report studies is misclassification when identifying and then filtering non-bug reports from the bug report repository. Having to filter out unrelated reports wastes time in identifying actual bug reports, and this escalates costs as extra maintenance and effort are required to triage and fix bugs. Therefore, this issue has been seriously studied and is addressed here. To tackle this problem, this study proposes a method of automatically identifying non-bug reports in the bug report repository using classification techniques. Three points are considered here. First, the bug report features used are unigram and CamelCase, where CamelCase words are used for feature expansion. Second, five term weighting schemes are compared to determine an appropriate term weighting scheme for this task. Lastly, the support vector machine (SVM) family i.e. binary-class SVM, one class SVM based on Scholkopf methodology and support vector data description (SVDD) are used as the main mechanisms for modeling non-bug report identifiers. After testing by recall, precision, and F1, the results demonstrate the efficiency of identifying non-bug reports in the bug report repository. Our results may be acceptable after comparing to the previous well-known studies, and the performance of non-bug report identifiers with tf-igm and modified tf-icf weighting schemes for both Scolkopf methodology and SVDD methods yielded the best value when compared to others.

Combination of a DAE-CNN and OC-SVDD for intrusion detection

The extensive use of the internet has favoured the emergence of intrusion detection systems (IDSs). The detection of malicious events requires the learning of the patterns representing the attacks. Meanwhile, new threats appear regularly. Thus, it is crucial to develop IDSs that do not depend on malicious patterns. In this paper, we propose an IDS based on the deep auto-encoder (DAE) algorithm for feature extraction, and on the one-class support vector data description (OC-SVDD) method for intrusion detection. The DAE is trained exclusively on normal patterns and is expected to extract robust features representing the normal traffic. The OC-SVDD is trained based on these features, thus, during the test stage, malicious events are classified as outliers. We report experiments, on the well-known NSL-KDD dataset. The experimental results show an accuracy of about 97.73% and prove the potential of the proposed.

A machine learning approach for modeling irregular regions with multiple owners in wind farm layout design

Wind farm development projects require a detailed survey of the eligible land. The land selected is often segmented into different region, each owned by different landowners with different land pricing. These regions are often complex shaped with irregular boundaries. Therefore, an efficient method for numerically modeling such irregular domains is needed. This work uses support vector data description (SVDD) to generate an analytical, continuous description of the irregular regions. Whereas other methods typically work well for modeling convex domains, the SVDD approach can be used to model irregular regions as a spherical boundary using various kernel mapping. It was demonstrated that the SVDD approach can be used to model any number of complex regions. An error analysis showed that the SVDD approach can construct accurate descriptions using a relatively small data set. The applicability of SVDD method in wind farm layout optimization is also demonstrated. The wind farm optimization study considered that the terrain is divided into several regions each owned by a different owner offering the land at a different price. Two different methods for considering the cost of the land are presented. The differences in optimal farm layouts using the two land cost models were also presented. In each case, the optimized wind farm layouts resulted in lower cost-of-energy relative to the reference wind farm. It was shown that the SVDD approach can also be used to restrict the placement of wind turbines in infeasible/restricted regions. The library for support vector data description was also made available to the public.

Deep anomaly detection for industrial systems: a case study

We explore the use of deep neural networks for anomaly detection of industrial systems where the data are multivariate time series measurements. We formulate the problem as a self-supervised learning where data under normal operation is used to train a deep neural network autoregressive model, i.e., use a window of time series data to predict future data values. The aim of such a model is to learn to represent the system dynamic behavior under normal conditions, while expect higher model vs. measurement discrepancies under faulty conditions. In real world applications, many control settings are categorical in nature. In this paper, vector embedding and joint losses are employed to deal with such situations. Both LSTM and CNN based deep neural network backbones are studied on the Secure Water Treatment (SWaT) testbed datasets. Also, Support Vector Data Description (SVDD) method is adapted to such anomaly detection settings with deep neural networks. Evaluation methods and results are discussed based on the SWaT dataset along with potential pitfalls.