One-class Support Vector Machine Research Articles

Anomaly detection in telemetry data using a jointly optimal one-class support vector machine with dictionary learning

Anomaly detection based on telemetry data is a major issue in satellite health monitoring, given that it can identify unusual or unexpected events to avoid serious accidents and ensure the safety and reliability of operations. This study proposes a jointly optimal one-class support vector machine with dictionary learning (JODL-OC) framework for multivariate anomaly detection in telemetry. Dictionary learning (DL) extracts the correlation and local dynamics of time series into a sparse matrix, and the one-class support vector machine (OCSVM) detects anomalies in the transformed sparse space. The optimisation objective of DL is modified by introducing a novel regularisation term called soft-boundary OCSVM, so that the JODL-OC seamlessly integrates DL, sparse representation, and classifier training into a unified model to jointly learn the features and decision boundary. Furthermore, we propose an iterative approach to solve the optimisation function and obtain the optimal JODL-OC framework for anomaly detection. A case analysis based on satellite antenna telemetry data reveals that the proposed method improves the detection precision, F1-score, FPR and FAR compared to other existing anomaly detection methods, particularly to illustrate the superior performance in correlation anomalies.

Detecting medial patellar luxation with ensemble deep convolutional neural network based on a single rear view image of the hindlimb

Medial patellar luxation (MPL) is a common orthopedic disease in dogs, which predisposes elderly and small-breed dogs. Unlike in humans, diagnosis in the early course of the disease is challenging because symptoms and joint-pain expression in canines are vague. Herein, we introduced a deep-learning system to diagnose MPL using a single rear-view hindlimb image. We believe that this is the first attempt to build a deep-learning system to diagnose MPL based on image analysis. Notably, 7689 images were collected from 2653 dogs in 30 private animal clinics between July 2021 and July 2022. Model performance was compared with ResNet50, VGG16, VGG19, Inception-V3, and veterinarian performance. For performance comparison, a professional veterinarian with > 10 years of experience selected images of 25 normal dogs and 25 dogs with MPL. The proposed model showed the highest performance, with 92.5% accuracy, whereas human experts showed an average accuracy of 55.2%. Therefore, our model can diagnose MPL using only a single rear-view hindlimb image. Furthermore, to solve the image uncertainty caused by the input image noise, we used a one-class SVM and ensemble learning methods to ensure model robustness. Our study will help diagnose MPL in clinical settings using a single rear-view hindlimb image.

Studying Imbalanced Learning for Anomaly-Based Intelligent IDS for Mission-Critical Internet of Things

Training-anomaly-based, machine-learning-based, intrusion detection systems (AMiDS) for use in critical Internet of Things (CioT) systems and military Internet of Things (MioT) environments may involve synthetic data or publicly simulated data due to data restrictions, data scarcity, or both. However, synthetic data can be unrealistic and potentially biased, and simulated data are invariably static, unrealistic, and prone to obsolescence. Building an AMiDS logical model to predict the deviation from normal behavior in MioT and CioT devices operating at the sensing or perception layer due to adversarial attacks often requires the model to be trained using current and realistic data. Unfortunately, while real-time data are realistic and relevant, they are largely imbalanced. Imbalanced data have a skewed class distribution and low-similarity index, thus hindering the model’s ability to recognize important features in the dataset and make accurate predictions. Data-driven learning using data sampling, resampling, and generative methods can lessen the adverse impact of a data imbalance on the AMiDS model’s performance and prediction accuracy. Generative methods enable passive adversarial learning. This paper investigates several data sampling, resampling, and generative methods. It examines their impacts on the performance and prediction accuracy of AMiDS models trained using imbalanced data drawn from the UNSW_2018_IoT_Botnet dataset, a publicly available IoT dataset from the IEEEDataPort. Furthermore, it evaluates the performance and predictability of these models when trained using data transformation methods, such as normalization and one-hot encoding, to cover a skewed distribution, data sampling and resampling methods to address data imbalances, and generative methods to train the models to increase the model’s robustness to recognize new but similar attacks. In this initial study, we focus on CioT systems and train PCA-based and oSVM-based AMiDS models constructed using low-complexity PCA and one-class SVM (oSVM) ML algorithms to fit an imbalanced ground truth IoT dataset. Overall, we consider the rare event prediction case where the minority class distribution is disproportionately low compared to the majority class distribution. We plan to use transfer learning in future studies to generalize our initial findings to the MioT environment. We focus on CioT systems and MioT environments instead of traditional or non-critical IoT environments due to the stringent low energy, the minimal response time constraints, and the variety of low-power, situational-aware (or both) things operating at the sensing or perception layer in a highly complex and open environment.

Adaptive Deep Feature Fusion for Continuous Authentication With Data Augmentation

Mobile devices are becoming increasingly popular and are playing significant roles in our daily lives. Insufficient security and weak protection mechanisms, however, cause serious privacy leakage of the unattended devices. To fully protect mobile device privacy, we propose ADFFDA, a novel mobile continuous authentication system using an Adaptive Deep Feature Fusion scheme for effective feature representation, and a transformer-based GAN for Data Augmentation, by leveraging smartphone built-in sensors of the accelerometer, gyroscope and magnetometer. Given the normalized sensor data, ADFFDA utilizes the transformer-based GAN consisting of a transformer-based generator and a CNN-based discriminator to augment the training data for CNN training. With the augmented data and the especially designed CNN based on the ghost module and ghost bottleneck, ADFFDA extracts deep features from the three sensors by the trained CNN, and exploits an adaptive-weighted concatenation method to adaptively fuse the CNN-extracted features. Based on the fused features, ADFFDA authenticates users by using the one-class SVM (OC-SVM) classifier. We evaluate the authentication performance of ADFFDA in terms of the efficiency of the transformer-based GAN, GAN-based data augmentation, CNN architecture, adaptive-weighted feature fusion, and OC-SVM classifier. The experimental results show that ADFFDA obtains the best authentication performance w.r.t representative approaches, by achieving a mean equal error rate of 0.01%.

Semi-Supervised Anomaly Detection of Dissolved Oxygen Sensor in Wastewater Treatment Plants.

As the world progresses toward a digitally connected and sustainable future, the integration of semi-supervised anomaly detection in wastewater treatment processes (WWTPs) promises to become an essential tool in preserving water resources and assuring the continuous effectiveness of plants. When these complex and dynamic systems are coupled with limited historical anomaly data or complex anomalies, it is crucial to have powerful tools capable of detecting subtle deviations from normal behavior to enable the early detection of equipment malfunctions. To address this challenge, in this study, we analyzed five semi-supervised machine learning techniques (SSLs) such as Isolation Forest (IF), Local Outlier Factor (LOF), One-Class Support Vector Machine (OCSVM), Multilayer Perceptron Autoencoder (MLP-AE), and Convolutional Autoencoder (Conv-AE) for detecting different anomalies (complete, concurrent, and complex) of the Dissolved Oxygen (DO) sensor and aeration valve in the WWTP. The best results are obtained in the case of Conv-AE algorithm, with an accuracy of 98.36 for complete faults, 97.81% for concurrent faults, and 98.64% for complex faults (a combination of incipient and concurrent faults). Additionally, we developed an anomaly detection system for the most effective semi-supervised technique, which can provide the detection of delay time and generate a fault alarm for each considered anomaly.

An Unsupervised Gradient-Based Approach for Real-Time Log Analysis From Distributed Systems

We consider the problem of real-time log anomaly detection for distributed system with deep neural networks by unsupervised learning. There are two challenges in this problem, including detection accuracy and analysis efficacy. To tackle these two challenges, we propose GLAD, a simple yet effective approach mining for anomalies in distributed systems. To ensure detection accuracy, we exploit the gradient features in a well-calibrated deep neural network and analyze anomalous pattern within log files. To improve the analysis efficacy, we further integrate one-class support vector machine (SVM) into anomalous analysis, which significantly reduces the cost of anomaly decision boundary delineation. This effective integration successfully solves both accuracy and efficacy in real-time log anomaly detection. Also, since anomalous analysis is based upon unsupervised learning, it significantly reduces the extra data labeling cost. We conduct a series of experiments to justify that GLAD has the best comprehensive performance balanced between accuracy and efficiency, which implies the advantage in tackling practical problems. The results also reveal that GLAD enables effective anomaly mining and consistently outperforms state-of-the-art methods on both recall and F1 scores.

Long-Short-Term-Memory-Based Deep Stacked Sequence-to-Sequence Autoencoder for Health Prediction of Industrial Workers in Closed Environments Based on Wearable Devices.

To reduce the risks and challenges faced by frontline workers in confined workspaces, accurate real-time health monitoring of their vital signs is essential for improving safety and productivity and preventing accidents. Machine-learning-based data-driven methods have shown promise in extracting valuable information from complex monitoring data. However, practical industrial settings still struggle with the data collection difficulties and low prediction accuracy of machine learning models due to the complex work environment. To tackle these challenges, a novel approach called a long short-term memory (LSTM)-based deep stacked sequence-to-sequence autoencoder is proposed for predicting the health status of workers in confined spaces. The first step involves implementing a wireless data acquisition system using edge-cloud platforms. Smart wearable devices are used to collect data from multiple sources, like temperature, heart rate, and pressure. These comprehensive data provide insights into the workers' health status within the closed space of a manufacturing factory. Next, a hybrid model combining deep learning and support vector machine (SVM) is constructed for anomaly detection. The LSTM-based deep stacked sequence-to-sequence autoencoder is specifically designed to learn deep discriminative features from the time-series data by reconstructing the input data and thus generating fused deep features. These features are then fed into a one-class SVM, enabling accurate recognition of workers' health status. The effectiveness and superiority of the proposed approach are demonstrated through comparisons with other existing approaches.

BGP Dataset-Based Malicious User Activity Detection Using Machine Learning

Recent advances in the Internet and digital technology have brought a wide variety of activities into cyberspace, but they have also brought a surge in cyberattacks, making it more important than ever to detect and prevent cyberattacks. In this study, a method is proposed to detect anomalies in cyberspace by consolidating BGP (Border Gateway Protocol) data into numerical data that can be trained by machine learning (ML) through a tokenizer. BGP data comprise a mix of numeric and textual data, making it challenging for ML models to learn. To convert the data into a numerical format, a tokenizer, a preprocessing technique from Natural Language Processing (NLP), was employed. This process goes beyond merely replacing letters with numbers; its objective is to preserve the patterns and characteristics of the data. The Synthetic Minority Over-sampling Technique (SMOTE) was subsequently applied to address the issue of imbalanced data. Anomaly detection experiments were conducted on the model using various ML algorithms such as One-Class Support Vector Machine (One-SVM), Convolutional Neural Network–Long Short-Term Memory (CNN–LSTM), Random Forest (RF), and Autoencoder (AE), and excellent performance in detection was demonstrated. In experiments, it performed best with the AE model, with an F1-Score of 0.99. In terms of the Area Under the Receiver Operating Characteristic (AUROC) curve, good performance was achieved by all ML models, with an average of over 90%. Improved cybersecurity is expected to be contributed by this research, as it enables the detection and monitoring of cyber anomalies from malicious users through BGP data.

Machine learning-based anomaly detection of groundwater microdynamics: case study of Chengdu, China

Detection of subsurface hydrodynamic anomalies plays a significant role in groundwater resource management and environmental monitoring. In this paper, based on data from the groundwater level, atmospheric pressure, and precipitation in the Chengdu area of China, a method for detecting outliers considering the factors affecting groundwater levels is proposed. By analyzing the factors affecting groundwater levels in the monitoring site and eliminating them, simplified groundwater data is obtained. Applying sl-Pauta (self-learning-based Pauta), iForest (Isolated Forest), OCSVM (One-Class SVM), and KNN to synthetic data with known outliers, testing and evaluating the effectiveness of 4 technologies. Finally, the four methods are applied to the detection of outliers in simplified groundwater levels. The results show that in the detection of outliers in synthesized data, the OCSVM method has the best detection performance, with a precision rate of 88.89%, a recall rate of 91.43%, an F1 score of 90.14%, and an AUC value of 95.66%. In the detection of outliers in simplified groundwater levels, a qualitative analysis of the displacement data within the field of view indicates that the outlier detection performance of iForest and OCSVM is better than that of KNN. The proposed method for considering the factors affecting groundwater levels can improve the efficiency and accuracy of detecting outliers in groundwater level data.

Detection of Electric Bicycle Indoor Charging for Electrical Safety: A NILM Approach

The increasing fire accidents caused by electric bicycle (EB) indoor charging have raised great concerns. To ensure residential electrical safety, EB is prohibited from charging indoors by regulations, but efficient monitoring of EB indoor charging behaviour in pratice still remains as a challenging problem. From the perspective of non-intrusive load monitoring (NILM), this paper proposes a holistic approach to detect the EB indoor charging behaviour. Our approach utilizes four key elements: (i) a multi-window based cumulative sum algorithm to capture the turning-on event from aggregate power; (ii) a twostage feature selection method to determine the optimal features to form an effective appliance signature for load identification; (iii) a one-class support vector machine classifier to identify whether the turned-on appliance is EB or other appliances in a semi-supervised way; (iv) a Raspberry-Pi based edge device to implement the overall NILM algorithm. The experiment results verify the effectiveness of our NILM based approach, which can obtain a high detection performance with a lowdimensional feature space, thus providing a cost-effective and high-performance solution for EB indoor charging detection.

Detecting outliers in GNSS position time series using machine learning techniques

The Global Navigation Satellite System (GNSS) position time series is applied in studies that require high-precision positioning, such as monitoring tectonic movements and Earth deformation. Outliers in GNSS position time series can significantly impact the accuracy of station positioning and movement parameters, leading to distorted data analysis outcomes. This study investigates the effectiveness of three machine learning techniques, including-Isolation Forest, One-Class Support Vector Machines (O-C SVM), and Local Outlier Factor (LOF) for outlier detection in GNSS position time series, with a specific focus on the SYNT model where outliers account for a substantial proportion (15%). Through comprehensive analysis, our results highlight the exceptional performance of the Isolation Forest method. It demonstrates remarkable accuracy in identifying outliers, effectively detecting the majority of them, and achieving an area under the ROC curve close to 1. In contrast, the LOF method performs less effectively in outlier detection, while the O-C SVM method displays relatively higher accuracy in identifying normal data points. These findings emphasize the significant advantages of leveraging machine learning approaches in processing continuous GNSS measurement data. By effectively identifying and handling outliers, these techniques enhance the accuracy and reliability of data analysis in GNSS position time series, ultimately establishing their superiority in the field of data analysis.

Deep Autoencoder-Based Integrated Model for Anomaly Detection and Efficient Feature Extraction in IoT Networks

The intrusion detection system (IDS) is a promising technology for ensuring security against cyber-attacks in internet-of-things networks. In conventional IDS, anomaly detection and feature extraction are performed by two different models. In this paper, we propose a new integrated model based on deep autoencoder (AE) for anomaly detection and feature extraction. Firstly, AE is trained based on normal network traffic and used later to detect anomalies. Then, the trained AE model is employed again to extract useful low-dimensional features for anomalous data without the need for a feature extraction training stage, which is required by other methods such as principal components analysis (PCA) and linear discriminant analysis (LDA). After that, the extracted features are used by a machine learning (ML) or deep learning (DL) classifier to determine the type of attack (multi-classification). The performance of the proposed unified approach was evaluated on real IoT datasets called N-BaIoT and MQTTset, which contain normal and malicious network traffics. The proposed AE was compared with other popular anomaly detection techniques such as one-class support vector machine (OC-SVM) and isolation forest (iForest), in terms of performance metrics (accuracy, precision, recall, and F1-score), and execution time. AE was found to identify attacks better than OC-SVM and iForest with fast detection time. The proposed feature extraction method aims to reduce the computation complexity while maintaining the performance metrics of the multi-classifier models as much as possible compared to their counterparts. We tested the model with different ML/DL classifiers such as decision tree, random forest, deep neural network (DNN), conventional neural network (CNN), and hybrid CNN with long short-term memory (LSTM). The experiment results showed the capability of the proposed model to simultaneously detect anomalous events and reduce the dimensionality of the data.

Comparing Quality Control Procedures Based on Minimum Covariance Determinant and One-Class Support Vector Machine Methods of Aircraft Meteorological Data Relay Data Assimilation in a Binary Typhoon Forecasting Case

This study investigates the impact of assimilating Aircraft Meteorological Data Relay (AMDAR) observations on the prediction of two typhoons, Nesat and Haitang (2017), using the Gridpoint Statistical Interpolation (GSI) assimilation system and the Weather Research and Forecasting (WRF) model. Two quality control (QC) methods, Minimum Covariance Determinant (MCD) and one-class Support Vector Machine (OCSVM), were employed to perform QC on the AMDAR observations before data assimilation. The QC results indicated that both methods significantly reduced kurtosis, skewness, and discrepancies between the AMDAR data and the reanalysis data. The data distribution after applying the MCD-QC method exhibited a closer resemblance to a Gaussian distribution. Four numerical experiments were conducted to assess the impact of different AMDAR data qualities on typhoon forecasting, including a control experiment without data assimilation (EXP-CNTL), assimilating all AMDAR observations (EXP-RAW), assimilating observations after applying MCD-QC (EXP-MCD), and assimilating observations after applying OCSVM-QC (EXP-SVM). The results demonstrated that using AMDAR data in assimilation improved the track and intensity prediction of the typhoons. Furthermore, utilizing QC before assimilation enhanced the performance of track forecasting prediction, with EXP-MCD showing the best performance. As for intensity prediction, the three assimilation experiments exhibited varying strengths and weaknesses at different times, with EXP-MCD showing smaller intensity forecast errors on average.

High-Resolution National-Scale Mapping of Paddy Rice Based on Sentinel-1/2 Data

Rice has always been one of the major food sources for human beings, and the monitoring and planning of cultivation areas to maintain food security and achieve sustainable development is critical for this crop. Traditional manual ground survey methods have been recognized as being laborious, while remote-sensing technology can perform the accurate mapping of paddy rice due to its unique data acquisition capabilities. The recently emerged Google Earth Engine (GEE) cloud-computing platform was found to be capable of storing and computing the resources required for the rapid processing of massive quantities of remote-sensing data, thereby revolutionizing traditional analysis patterns and offering unique advantages for large-scale crop mapping. Since the phenology of paddy rice depends on local climatic conditions, and considering the vast expanse of China with its outstanding geospatial heterogeneity, a zoning strategy was proposed in this study to separate the monsoon climate zone of China into two regions based on the Qinling Mountain–Huaihe River Line (Q-H Line), while discrepant basic data and algorithms have been adopted to separately map mid-season rice nationwide. For the northern regions, optical indices have been calculated based on Sentinel-2 images, growth spectral profiles have been constructed to identify phenological periods, and rice was mapped using One-Class Support Vector Machine (OCSVM); for the southern regions, microwave sequences have been constructed based on Sentinel-1 images, and rice was mapped using Random Forest (RF). By applying this methodological system, mid-season rice at 10 m spatial resolution was mapped on the GEE for the entire Chinese monsoon region in 2021. According to the accuracy evaluation coefficients and publicly released local statistical yearbook data, the relative error of the mapped areas in each province was limited to 10%, and the overall accuracy exceeded 85%. The results could indicate that mid-season rice can be mapped more accurately and efficiently on a China-wide scale with relatively few samples based on the proposed zoning strategy and mapping methods. By adjusting the parameters, the time interval for mapping could also be further extended. The powerful cloud-computing competence of the GEE platform was used to map rice on a large spatial scale, and the results can help governments to ascertain the distribution of mid-season rice across the country in a short-term period, which would be well suited to meeting the increasingly efficient and fine-grained decision-making and management requirements.

A one-class support vector machine for detecting valve stiction

In industrial processes, control valve stiction is known to be one of the primary causes for poor control loop performance. Stiction introduces oscillatory behaviour in the process, leading to increased energy consumption, variations in product quality, shortened equipment lifespan and a reduction in overall plant profitability. Several detection algorithms using routine operating data have been developed over the last few decades. However, with the exception of a handful of recent publications, few attempts to apply classical supervised learning techniques have been published thus far. In this work, principal component analysis, linear discriminant analysis and a one-class support vector machine are trained to detect stiction using time series features as input. These features are extracted from the data using the tsfresh package for Python. The training data consists of simulated stiction examples generated using the XCH stiction model as well as other sources of oscillation. The classifier is subsequently benchmarked against closed-loop stiction data collected in an industrial setting, with performance exceeding that of existing methods.

Single-cell scattering and auto-fluorescence-based fast antibiotic susceptibility testing for gram-negative and gram-positive bacteria.

In this study, we assess the scattering of light and auto-fluorescence from single bacterial cells to address the challenge of fast (<2 h), label-free phenotypic antimicrobial susceptibility testing (AST). Label-free flow cytometry is used for monitoring both the respiration-related auto-fluorescence in two different fluorescence channels corresponding to FAD and NADH, and the morphological and structural information contained in the light scattered by individual bacteria during incubation with or without antibiotic. Large multi-parameter data are analyzed using dimensionality reduction methods, based either on a combination of 2D binning and Principal Component Analysis, or with a one-class Support Vector Machine approach, with the objective to predict the Susceptible or Resistant phenotype of the strain. For the first time, both Escherichia coli (Gram-negative) and Staphylococcus epidermidis (Gram-positive) isolates were tested with a label-free approach, and, in the presence of two groups of bactericidal antibiotic molecules, aminoglycosides and beta-lactams. Our results support the feasibility of label-free AST in less than 2 h and suggest that single cell auto-fluorescence adds value to the Susceptible/Resistant phenotyping over single-cell scattering alone, in particular for the mecA+ Staphylococcus (i.e., resistant) strains treated with oxacillin.

Fraud Detection in Enterprise Resource Planning Systems Using One-Class Support Vector Machine Combined with Convolutional Neural Network: The Case of Spor Istanbul

Fraud Detection in Enterprise Resource Planning Systems Using One-Class Support Vector Machine Combined with Convolutional Neural Network: The Case of Spor Istanbul

Unity is strength: Improving the detection of adversarial examples with ensemble approaches

A key challenge in computer vision and deep learning is the definition of robust strategies for the detection of adversarial examples. In this work, we propose the adoption of ensemble approaches to leverage the effectiveness of multiple detectors in exploiting distinct properties of the input data. To this end, the ENsemble Adversarial Detector (ENAD) framework integrates scoring functions from state-of-the-art detectors based on Mahalanobis distance, Local Intrinsic Dimensionality, and One-Class Support Vector Machines, which process the hidden features of deep neural networks. ENAD is designed to ensure high standardization and reproducibility to the computational workflow.Extensive tests on benchmark datasets, models and adversarial attacks show that ENAD outperforms all competing methods in the large majority of settings. The improvement over the state-of-the-art and the intrinsic generality of the framework, which allows one to easily extend ENAD to include any set of detectors and integration strategies, set the foundations for the new area of ensemble adversarial detection.

Welding fault detection and diagnosis using one-class SVM with distance substitution kernels and random convolutional kernel transform

Welding fault detection and diagnosis using one-class SVM with distance substitution kernels and random convolutional kernel transform

Anomaly Detection with Transformer in Face Anti-spoofing

Transformers are emerging as the new gold standard in various computer vision applications, and have already been used in face anti-spoofing demonstrating competitive performance. In this paper, we propose a network with the ViT transformer and ResNet as the backbone for anomaly detection in face anti-spoofing and compare the performance of various one-class classifiers at the end of the pipeline, such as one-class SVM, Isolation Forest, and decoders. Test results on the RA and SiW databases show the proposed approach to be competitive as an anomaly detection method for face anti-spoofing.