One-class Support Vector Machine Research Articles

A Hybrid Algorithm Incorporating Vector Quantization and One-Class Support Vector Machine for Industrial Anomaly Detection

Anomaly detection plays an important role in industry, especially in ensuring system safety and product quality. Due to the unavailability of anomalous data in many practical cases, anomaly detection is usually solved by one-class classification (OCC) methods using only normal data. As a classical OCC method, one-class support vector machine (OCSVM) is a popular discriminative approach for anomaly detection, which detects abnormal data points by establishing a decision boundary in the kernel space. However, the performance of OCSVM heavily relies on kernel parameters, whose selection is not trivial for anomaly detection problems. Moreover, for some uneven and complex data distributions, different data regions may have quite different densities and shapes, making it difficult for OCSVM to obtain good boundaries in all regions using a global kernel parameter. To address the above two issues, in this article, we propose a hybrid algorithm incorporating vector quantization and OCSVM (VQ-OCSVM). Specifically, vector quantization is used to extract distribution information of normal data, and the results are used to construct an explicit mapping function to map data into a high-dimensional feature space. Then, OCSVM is performed in the feature space to build a classifier. By introducing the explicit mapping into OCSVM, the proposed method can effectively bypass the kernel parameter selection problem of the classical OCSVM method. Furthermore, the constructed mapping carries the data distribution information, and the VQ-OCSVM model can be regarded as an integration of generative learning and discriminative learning. The complementary properties of these two paradigms make the proposed VQ-OCSVM algorithm have better generalization capacity for complex data distribution. Both qualitative and quantitative experimental results demonstrate the effectiveness and advantages of the proposed method.

Machine Learning for Ensuring Data Integrity in Salesforce Applications.

This study investigates the application of machine learning algorithms to enhance data integrity within Salesforce applications, addressing the challenge of detecting anomalies in complex CRM datasets. The research aims to evaluate the effectiveness of various machine learning models—specifically Isolation Forest, One-Class SVM, and Autoencoders—in identifying data irregularities and improving overall data quality. Utilizing a dataset of 10,000 records from Salesforce, the study involved preprocessing the data, implementing the ML models, and assessing their performance using metrics such as Precision, Recall, and F1 Score. Major findings indicate that machine learning models significantly outperform traditional anomaly detection methods, with Autoencoders demonstrating superior performance in handling high-dimensional data. The implementation of these models resulted in notable improvements in data accuracy and reduced error rates. The study concludes that integrating machine learning into CRM systems can substantially enhance data integrity, offering valuable insights for both theoretical research and practical applications. Future research should explore additional algorithms and real-world deployment challenges

Hyperparameter Tuning of OC-SVM for Industrial Gas Turbine Anomaly Detection

Gas turbine failure diagnosis is performed in this work based on seven types of tag data consisting of a total of 7976 data. The data consist of about 7000 normal data and less than 500 abnormal data. While normal data are easy to extract, failure data are difficult to extract. So, this study mainly is composed of normal data and a one-class support vector machine (OC-SVM) is used here, which has an advantage in classification accuracy performance. To advance the classification performance, four hyperparameter tuning (manual search, grid search, random search, Bayesian optimization) methods are applied. To analyze the performance of each technique, four evaluation indicators (accuracy, precision, recall, and F-1 score) are used. As a result, about 54.3% of the initial failure diagnosis performance is improved up to 64.88% through the advanced process in terms of accuracy.

SAOCNN: Self-Attention and One-Class Neural Networks for Hyperspectral Anomaly Detection

Hyperspectral anomaly detection is a popular research direction for hyperspectral images; however, it is problematic because it separates the background and anomaly without prior target information. Currently, deep neural networks are used as an extractor to mine intrinsic features in hyperspectral images, which can be fed into separate anomaly detection methods to improve their performances. However, this hybrid approach is suboptimal because the subsequent detector is unable to drive the data representation in hidden layers, which makes it a challenge to maximize the capabilities of deep neural networks when extracting the underlying features customized for anomaly detection. To address this issue, a novel unsupervised, self-attention-based, one-class neural network (SAOCNN) is proposed in this paper. SAOCNN consists of two components: a novel feature extraction network and a one-class SVM (OC-SVM) anomaly detection method, which are interconnected and jointly trained by the OC-SVM-like loss function. The adoption of co-training updates the feature extraction network together with the anomaly detector, thus improving the whole network’s detection performance. Considering that the prominent feature of an anomaly lies in its difference from the background, we designed a deep neural extraction network to learn more comprehensive hyperspectral image features, including spectral, global correlation, and local spatial features. To accomplish this goal, we adopted an adversarial autoencoder to produce the residual image with highlighted anomaly targets and a suppressed background, which is input into an improved non-local module to adaptively select the useful global information in the whole deep feature space. In addition, we incorporated a two-layer convolutional network to obtain local features. SAOCNN maps the original hyperspectral data to a learned feature space with better anomaly separation from the background, making it possible for the hyperplane to separate them. Our experiments on six public hyperspectral datasets demonstrate the state-of-the-art performance and superiority of our proposed SAOCNN when extracting deep potential features, which are more conducive to anomaly detection.

A Turboshaft Aeroengine Fault Detection Method Based on One-Class Support Vector Machine and Transfer Learning

A Turboshaft Aeroengine Fault Detection Method Based on One-Class Support Vector Machine and Transfer Learning

Distributed Online Anomaly Detection for Virtualized Network Slicing Environment

As the network slicing is one of the critical enablers in communication networks, one anomalous physical node (PN) or physical link (PL) in substrate networks that carries multiple virtual network elements can cause significant performance degradation of multiple network slices. To recover the substrate networks from anomaly within a short time, rapid and accurate identification of whether or not the anomaly exists in PNs and PLs is vital. Online anomaly detection methods that can analyze system data in real-time are preferred. Besides, as virtual nodes and links mapped to PNs and PLs are scattered in multiple slices, the distributed detection modes are required to adapt to the virtualized environment. According to those requirements, in this paper, we first propose a distributed online PN anomaly detection algorithm based on a decentralized one-class support vector machine (OCSVM), which is realized through analyzing real-time measurements of virtual nodes mapped to PNs in a distributed manner. Specifically, to decouple the OCSVM objective function, we transform the original problem to a group of decentralized quadratic programming problems by introducing the consensus constraints. The alternating direction method of multipliers is adopted to achieve the solution for the distributed online PN anomaly detection. Next, by utilizing the correlation of measurements between neighbor virtual nodes, another distributed online PL anomaly detection algorithm based on the canonical correlation analysis is proposed. The network only needs to store covariance matrices and mean vectors of current data to calculate the canonical correlation vectors for real-time PL anomaly analysis. The simulation results on both synthetic and real-world network datasets show the effectiveness and robustness of the proposed distributed online anomaly detection algorithms.

Using artificial intelligence to detect human errors in nuclear power plants: A case in operation and maintenance

Human error (HE) is an important concern in safety-critical systems such as nuclear power plants (NPPs). HE has played a role in many accidents and outage incidents in NPPs. Despite the increased automation in NPPs, HE remains unavoidable. Hence, the need for HE detection is as important as HE prevention efforts. In NPPs, HE is rather rare. Hence, anomaly detection, a widely used machine learning technique for detecting rare anomalous instances, can be repurposed to detect potential HE. In this study, we develop an unsupervised anomaly detection technique based on generative adversarial networks (GANs) to detect anomalies in manually collected surveillance data in NPPs. More specifically, our GAN is trained to detect mismatches between automatically recorded sensor data and manually collected surveillance data, and hence, identify anomalous instances that can be attributed to HE. We test our GAN on both a real-world dataset and an external dataset obtained from a testbed, and we benchmark our results against state-of-the-art unsupervised anomaly detection algorithms, including one-class support vector machine and isolation forest. Our results show that the proposed GAN provides improved anomaly detection performance. Our study is promising for the future development of artificial intelligence based HE detection systems.

Data-Driven Method for Detecting Flight Trajectory Deviation Anomaly

With the rapid growth of air traffic, it often happens that aircraft deviate from the original flight plan during actual flight. This paper proposes an anomaly detection method for aircraft trajectory deviation to realize single-point and successive multipoint anomaly detection from a data-driven perspective. Given the one-to-many relationship between reporting points of planned and real trajectories, a matching algorithm is used to match these points. Four trajectory deviation features (which are the position deviation, distance deviation, altitude deviation, and flight stage) are defined. On this basis, a one-class support vector machine is trained to detect single-point anomalies using the deviation features as input. Furthermore, successive multipoint anomaly detection of the aircraft is realized by considering the deviation of successive segments of the trajectory. Taking the flights taking off and landing at four Chinese hub airports as examples, the proposed method obtained an F score, which is a balance of the precision and recall, over 0.92, indicating it achieves high accuracy for anomaly detection.

Machine learning for detecting centre-level irregularities in randomized controlled trials: A pilot study

Centralized statistical monitoring is sometimes employed as an alternative to onsite monitoring for randomized control trials. Current central monitoring methods have limitations, in that they are relatively resource intensive and do not necessarily generalize to studies where an irregularity pattern has not been observed before. Machine learning has been effective in detecting irregularities in industries such as finance and manufacturing, but to date none have been applied to clinical trials. We conducted a pilot study for the use of machine learning to identify center-level irregularities in data from multicenter clinical trials.We employed unsupervised machine learning methods, which do not rely on labelled data, and therefore allow for the automated discovery of previously unseen irregularity patterns while maintaining flexibility when applied to new data with different structures. This pilot study employs unsupervised machine learning to compute distance matrices between centres, which we used to produce centre-level continuous features. We then used a one-class support vector machine to learn the underlying distribution of each data set to identify data that was substantially different from these distributions.We evaluated our approach against current automatable centralized monitoring methods on two trials with known irregularities. While current approaches performed well on one trial (AUROC 0.752 for monitoring vs. 0.584 for machine learning), our techniques performed substantially better on the other (AUROC 0.140 for monitoring vs 0.728 for machine learning).The results of this pilot study suggest both the feasibility and the potential value of a machine learning-based approach to irregularity detection in RCTs.

Radiomic features of amygdala nuclei and hippocampus subfields help to predict subthalamic deep brain stimulation motor outcomes for Parkinson's disease patients.

Background and purposeThe aim of the study is to predict the subthalamic nucleus (STN) deep brain stimulation (DBS) outcomes for Parkinson’s disease (PD) patients using the radiomic features extracted from pre-operative magnetic resonance images (MRI).MethodsThe study included 34 PD patients who underwent DBS implantation in the STN. Five patients (15%) showed poor DBS motor outcome. All together 9 amygdalar nuclei and 12 hippocampus subfields were segmented using Freesurfer 7.0 pipeline from pre-operative MRI images. Furthermore, PyRadiomics platform was used to extract 120 radiomic features for each nuclei and subfield resulting in 5,040 features. Minimum Redundancy Maximum Relevance (mRMR) feature selection method was employed to reduce the number of features to 20, and 8 machine learning methods (regularized binary logistic regression (LR), decision tree classifier (DT), linear discriminant analysis (LDA), naive Bayes classifier (NB), kernel support vector machine (SVM), deep feed-forward neural network (DNN), one-class support vector machine (OC-SVM), feed-forward neural network-based autoencoder for anomaly detection (DNN-A)) were applied to build the models for poor vs. good and very good STN-DBS motor outcome prediction.ResultsThe highest mean prediction accuracy was obtained using regularized LR (96.65 ± 7.24%, AUC 0.98 ± 0.06) and DNN (87.25 ± 14.80%, AUC 0.87 ± 0.18).ConclusionThe results show the potential power of the radiomic features extracted from hippocampus and amygdala MRI in the prediction of STN-DBS motor outcomes for PD patients.

Abnormal Gait and Tremor Detection in the Elderly Ambulatory Behavior Using an IoT Smart Cane Device

In this paper, a novel approach for abnormal gait and tremor detection using a smart walking cane is introduced. Periodic muscle movement associated with Parkinson’s disease, such as arm shaking, vibrating arm, trembling fingers, rhythmic wrist movements, normal and abnormal walking pattern, was learned and classified with linear discriminant analysis. Although detecting symptoms related to disease with walking sticks might look trivial at first, throughout history, a cane or walking stick has been used as an assistive device to aid in ambulating, especially in the elderly and disabled, so embedding smart devices (that can learn ambulating pattern and detect anomalies associated with it) in the cane will help in early detection of diseases and facilitate early intervention. This approach is non-intrusive, and privacy issues being experienced in visual models do not arise, as users do not need to wear any special bracelet or wrist monitoring, and they only need to pick up the cane when they wish to move. The simplicity and efficient usage of a technique for detecting ambulatory anomalies is also demonstrated in this research. We extracted step counts, fall data and other valuable features from the cane, and detected anomalies by using isolation forest and one-class support vector machine (SVM) methods. Falls were detected easily and naturally with the cane, which had different alert modes (a soft alert when the cane lost equilibrium and was picked up within 15 s, and a strong alert otherwise). Intervention systems are proposed to forestall and limit the possibility of a type 2 error.

Evaluation of machine learning approaches for surface water monitoring using Sentinel-1 data

The monitoring of mosquito breeding habitats requires the production of surface water maps on a regular basis and at a high-resolution using mapping algorithms. To map surface water, several machine learning (ML) algorithms were evaluated, taking advantage of frequently available synthetic aperture radar imagery from the Sentinel-1 mission with a 10-m spatial resolution and a large dataset of field observations of the water state (inundated/dry) in rice paddies and wetlands. One-class support vector machine, one-class self-organizing map, and multilayer perceptron with automatic relevance determination (MLP-ARD) algorithms were trained and assessed to examine their accuracy in detecting surface water. Results show the robustness of the MLP-ARD algorithm, which provides an overall accuracy of 0.974 for a single date and 0.892 for a 5-month study period from May to September. The accuracy of water detection was found to be mainly affected by the presence of dense and high vegetation in inundated fields and the presence of floating vegetation or algae.

Collision Detection for Robot Manipulators Using Unsupervised Anomaly Detection Algorithms

Collaborative robot manipulators operating in dynamic and unstructured environments shared with humans require fast and accurate detection of collisions. When using model-based detection methods without joint torque sensors, proper treatment for friction in the motors is required, such as accurate modeling of friction parameters. In our previous work, we have presented two supervised learning-based detection methods, which successfully compensate for uncertain dynamic effects, including unmodeled friction. However, robot collision data are required for training and only the scenarios that are learned as collisions can be robustly detected. In this article, we present two collision detection methods using unsupervised anomaly detection algorithms—a one-class support vector machine, and an autoencoder—for robot manipulators. Only the motor current measurements together with a robot dynamics model are required while no additional external sensors, friction modeling, or manual tuning of multiple detection thresholds are needed. Extensive experiments with a robot collision dataset collected with a six-joint robot manipulator validate that our unsupervised detection algorithms can robustly detect a variety of hard and soft collisions with very light computation using only the data for collision-free motions and without the knowledge of every possible type of collision that can occur.

An outlier detection-based method for artifact removal of few-channel EEGs

Objective. The electroencephalogram (EEG) is one of the most important brain-imaging tools. The few-channel EEG is more suitable and affordable for practical use as a wearable device. Removing artifacts from collected EEGs is a prerequisite for accurately interpreting brain function and state. Previous studies proposed methods combining signal decomposition with the blind source separation (BSS) algorithms, but most of them used threshold-based criteria for artifact rejection, resulting in a lack of effectiveness in removing specific artifacts and the excessive suppression of brain activities. In this study, we proposed an outlier detection-based method for artifact removal under the few-channel condition. Approach. The underlying components (sources) were extracted using the decomposition-BSS schema. Based on our assumptions that in the feature space, the artifact-related components are dispersed, while the components related to brain activities are closely distributed, the artifact-related components were identified and rejected using one-class support vector machine. The assumptions were validated by visualizing the distribution of clusters of components. Main results. In quantitative analyses with semisimulated data, the proposed method outperformed the threshold-based methods for various artifacts, including muscle artifact, ocular artifact, and power line noise. With a real dataset and an event-related potential dataset, the proposed method demonstrated good performance in real-life situations. Significance. This study provided a fully data-driven and adaptive method for removing various artifacts in a single process without excessive suppression of brain activities.

Adaptive Physical Layer Authentication Using Machine Learning With Antenna Diversity

The heterogeneous characteristics of wireless mobile networks within the Internet of things (IoT) create authentication challenges due to the large number of devices with diverse requirements and capabilities. Physical layer authentication (PLA) can provide solutions for this heterogeneous environment using wireless channel attributes. In the paper, an adaptive lightweight physical layer authentication (ALPLA) scheme is proposed using machine learning (ML). Antenna diversity at the receiver is exploited to increase the number of features to achieve a high authentication rate (AR). A one-class classifier support vector machine (OCC-SVM) is used with the magnitude and real and imaginary parts of the received signal at each receive antenna as features. One-class classification is a ML technique for outlier and anomaly detection which uses only legitimate training data. The sounding reference signal (SRS) in the 5G uplink radio frame is employed to obtain the features. The proposed scheme is evaluated in an urban environment under different mobility conditions. Results are presented which show that this scheme provides a high AR with sufficient antenna diversity. Further it is superior to other approaches in the literature.

Unsupervised Learning Approach for Detection and Localization of Structural Damage using Output-only Measurements

Interrogation of the vibration data collected from the sensors embedded throughout the structure without relying on a finite element model of the system for monitoring the health of structural systems has received significant attention in the recent years especially with the current advancements in sensor technology. The data-driven methods explored within this context falls into the realm of statistical pattern recognition field requiring extraction of damage detection features and a statistical decision-making process for identification of damage. Machine learning algorithms provide statistical means for making such decisions. In this study, an unsupervised machine learning approach, one-class support vector machine (OC-SVM), requiring training data only from the undamaged state of the structure is explored for damage detection purposes. The coefficients of the autoregressive (AR) model are extracted as damage sensitive features and used as the required training data. The trained classifier is then used with the data obtained from the same structure at different damage states for classification. Damage detection in the form of recognizing outliers or anomalies not belonging to the target class, is followed by damage localization within the given sensor resolution using statistical means. Numerical simulations are performed on a truss and a beam structure with several damage scenarios illustrating the capabilities and the limitations of the proposed approach.

An efficient method for autoencoder based outlier detection

Unsupervised Learning is widely used approach for outlier detection because non-availability of training dataset in various domains (especially, in evolving domains). Approaches like clustering-based, distance-based, density-based outlier detection methods have been proposed over the last several years. Recently, outlier detection using deep learning has drawn attention of researchers. Deep learning-based unsupervised techniques (autoencoder) minimize the reconstruction error using each data instance in the dataset and subsequently, data points with higher reconstruction error are treated as outlier points. However, autoencoder based model overestimates the reconstruction error for normal points whereas it is underestimated for outlier points. As a result, genuine outliers are missed by this approach.We propose two techniques to address the issue of reconstruction error stated earlier. Main idea of our techniques is to compute reconstruction error only using ‘normal points’. In the proposed techniques, we identify probable outliers utilizing the clustering approaches intelligently and subsequently, we do not include them in the minimization process of reconstruction error. We exploit recently recognized clustering approach Density Peak Clustering (DPC) to identify the probable outlier points based on density and distance to the higher density points. However, DPC has inherent drawback of setting threshold which plays important role in deciding density. Therefore, Self Organizing Map (SOM) is exploited as another clustering approach in this article. Subsequently, we conducted experiments on synthetic as well as real world datasets and the results show that the proposed technique outperforms the popular existing deep learning model like RandNet, Boosting-based Autoencoder Ensemble method (BAE), One Class Support Vector Machine (OCSVM) and density-based algorithms like LOF, LDOF, INFLO, and RDOS.

New Interior-Point Approach for One- and Two-Class Linear Support Vector Machines Using Multiple Variable Splitting

Multiple variable splitting is a general technique for decomposing problems by using copies of variables and additional linking constraints that equate their values. The resulting large optimization problem can be solved with a specialized interior-point method that exploits the problem structure and computes the Newton direction with a combination of direct and iterative solvers (i.e. Cholesky factorizations and preconditioned conjugate gradients for linear systems related to, respectively, subproblems and new linking constraints). The present work applies this method to solving real-world binary classification and novelty (or outlier) detection problems by means of, respectively, two-class and one-class linear support vector machines (SVMs). Unlike previous interior-point approaches for SVMs, which were practical only with low-dimensional points, the new proposal can also deal with high-dimensional data. The new method is compared with state-of-the-art solvers for SVMs that are based on either interior-point algorithms (such as SVM-OOPS) or specific algorithms developed by the machine learning community (such as LIBSVM and LIBLINEAR). The computational results show that, for two-class SVMs, the new proposal is competitive not only against previous interior-point methods—and much more efficient than they are with high-dimensional data—but also against LIBSVM, whereas LIBLINEAR generally outperformed the proposal. For one-class SVMs, the new method consistently outperformed all other approaches, in terms of either solution time or solution quality.

Automatic detection of primary and metastatic lesions on cervicothoracic region and whole-body bone using a uniform machine-learnable approach for [18F]-FDG-PET/CT image analysis

We propose a method to detect primary and metastatic lesions with Fluorine−18 fluorodeoxyglucose (FDG) accumulation in the lung field, neck, mediastinum, and bony regions on the FDG-PET/CT images. To search for systemic lesions, various anatomical structures must be considered. The proposed method is addressed by using an extraction process for anatomical regions and a uniform lesion detection approach. The uniform approach does not utilize processes that reflect any region-specific anatomical aspects but has a machine-learnable framework. Therefore, it can work as a lesion detection process for a specific anatomical region if it machine-learns the specific region data. In this study, three lesion detection processes for the whole-body bone region, lung field, or neck-mediastinum region are obtained. These detection processes include lesion candidate detection and false positive (FP) candidate elimination. The lesion candidate detection is based on a voxel anomaly detection with a one-class support vector machine. The FP candidate elimination is performed using an AdaBoost classifier ensemble. The image features used by the ensemble are selected sequentially during training and are optimal for candidate classification. Three-fold cross-validation was used to detect performance with the 54 diseased FDG-PET/CT images. The mean sensitivity for detecting primary and metastatic lesions at 3 FPs per case was 0.89 with a 0.10 standard deviation (SD) in the bone region, 0.80 with a 0.10 SD in the lung field, and 0.87 with a 0.10 SD in the neck region. The average areas under the ROC curve were 0.887 with a 0.125 SD for detecting bone metastases, 0.900 with a 0.063 SD for detecting pulmonary lesions, and 0.927 with a 0.035 SD for detecting the neck-mediastinum lesions. These detection performances indicate that the proposed method could be applied clinically. These results also show that the uniform approach has high versatility for providing various lesion detection processes.

The Application of Computer Intelligence in the Cyber-Physical Business System Integration in Network Security.

In order to address the false alarm detection problem caused by the inability to identify the transgression scene pages in the process of horizontal transgression detection, this study proposes a deep learning-based LSTM-AutoEncoder unsupervised prediction model. The model uses long short-term memory network to build AutoEncoder, extracts text features of page response data of horizontal transgression scenario, and reconstructs text features to restore. Meanwhile, it counts the error between the restored result and the original page response, judges whether the detection result of horizontal transgression is false alarm according to the error threshold of unknown page, and tests the effectiveness of the model effect under real business data by comparing it with other two algorithms, one-class SVM and AutoEncoder, which provides security for enterprise network business. The results show that the LSTM-AutoEncoder model achieves a more balanced index in terms of accuracy, precision, recall, and F1-score in the case of MAE, which is 0.3% more and 0.2% more than the case of MSE in terms of recall and accuracy. It is concluded that the LSTM-AutoEncoder model is more in line with the real business requirements, and the simple model architecture selected for this study can reduce the complexity of the model, speed up the prediction time of the model in the application phase, and improve the performance of the detection software. This indicates that this study has some application prospects in network security.