One-class Learning Research Articles

Anomaly Detection in Self-Organizing Networks: Conventional Versus Contemporary Machine Learning

This paper presents a comparison of conventional and modern machine (deep) learning within the framework of anomaly detection in self-organizing networks. While deep learning has gained significant traction, especially in application scenarios where large volumes of data can be collected and processed, conventional methods may yet offer strong statistical alternatives, especially when using proper learning representations. For instance, support vector machines have previously demonstrated state-of-the-art potential in many binary classification applications and can be further exploited with different representations, such as one-class learning and data augmentation. We demonstrate for the first time, on a previously published and publicly available dataset, that conventional machine learning can outperform the previous state-of-the-art using deep learning by 15% on average across four different application scenarios. Our results further indicate that with nearly two orders of magnitude improvement in computational speed and an order of magnitude reduction in trainable parameters, conventional machine learning provides a robust alternative for 5G self-organizing networks especially when the execution and detection times are critical.

Lightweight Anomaly Detection Scheme Using Incremental Principal Component Analysis and Support Vector Machine.

Wireless Sensors Networks have been the focus of significant attention from research and development due to their applications of collecting data from various fields such as smart cities, power grids, transportation systems, medical sectors, military, and rural areas. Accurate and reliable measurements for insightful data analysis and decision-making are the ultimate goals of sensor networks for critical domains. However, the raw data collected by WSNs usually are not reliable and inaccurate due to the imperfect nature of WSNs. Identifying misbehaviours or anomalies in the network is important for providing reliable and secure functioning of the network. However, due to resource constraints, a lightweight detection scheme is a major design challenge in sensor networks. This paper aims at designing and developing a lightweight anomaly detection scheme to improve efficiency in terms of reducing the computational complexity and communication and improving memory utilization overhead while maintaining high accuracy. To achieve this aim, one-class learning and dimension reduction concepts were used in the design. The One-Class Support Vector Machine (OCSVM) with hyper-ellipsoid variance was used for anomaly detection due to its advantage in classifying unlabelled and multivariate data. Various One-Class Support Vector Machine formulations have been investigated and Centred-Ellipsoid has been adopted in this study due to its effectiveness. Centred-Ellipsoid is the most effective kernel among studies formulations. To decrease the computational complexity and improve memory utilization, the dimensions of the data were reduced using the Candid Covariance-Free Incremental Principal Component Analysis (CCIPCA) algorithm. Extensive experiments were conducted to evaluate the proposed lightweight anomaly detection scheme. Results in terms of detection accuracy, memory utilization, computational complexity, and communication overhead show that the proposed scheme is effective and efficient compared few existing schemes evaluated. The proposed anomaly detection scheme achieved the accuracy higher than 98%, with (nd) memory utilization and no communication overhead.

A Bayesian Approach to the Data Description Problem

In this paper, we address the problem of data description using a Bayesian framework. The goal of data description is to draw a boundary around objects of a certain class of interest to discriminate that class from the rest of the feature space. Data description is also known as one-class learning and has a wide range of applications. The proposed approach uses a Bayesian framework to precisely compute the class boundary and therefore can utilize domain information in form of prior knowledge in the framework. It can also operate in the kernel space and therefore recognize arbitrary boundary shapes. Moreover, the proposed method can utilize unlabeled data in order to improve accuracy of discrimination. We evaluate our method using various real-world datasets and compare it with other state of the art approaches of data description. Experiments show promising results and improved performance over other data description and one-class learning algorithms.

Plant Leaves Recognition Based on a Hierarchical One-Class Learning Scheme with Convolutional Auto-Encoder and Siamese Neural Network

In this paper, we propose a novel method for plant leaves recognition by incorporating an unsupervised convolutional auto-encoder (CAE) and Siamese neural network in a unified framework by considering Siamese as an alternative to the conventional loss of CAE. Rather than the conventional exploitation of CAE and Siamese, in our case we have proposed to extend CAE for a novel supervised scenario by considering it as one-class learning classifier. For each class, CAE is trained to reconstruct its positive and negative examples and Siamese is trained to distinguish the similarity and the dissimilarity of the obtained examples. On the contrary and asymmetric to the related hierarchical classification schemes which require pre-knowledge on the dataset being recognized, we propose a hierarchical classification scheme that doesn’t require such a pre-knowledge and can be employed by non-experts automatically. We cluster the dataset to assemble similar classes together. A test image is first assigned to the nearest cluster, then matched to one class from the classes that fall under the determined cluster using our novel one-class learning classifier. The proposed method has been evaluated on the ImageCLEF2012 dataset. Experimental results have proved the superiority of our method compared to several state-of-the art methods.

A graph-based approach for positive and unlabeled learning

Positive and Unlabeled Learning (PUL) uses unlabeled documents and a few positive documents for retrieving a set of “interest” documents from a text collection. Usually, PUL approaches are based on the vector space model. However, when dealing with semi-supervised learning for text classification or information retrieval, graph-based approaches have been proved to outperform vector space model-based approaches. So, in this article, a graph-based approach for PUL is proposed: Label Propagation for Positive and Unlabeled Learning (LP-PUL). The proposed framework consists of three steps: (i) building a similarity graph, (ii) identifying reliable negative documents, and (iii) performing label propagation to classify the remaining unlabeled documents as positive or negative. We carried out experiments to measure the impact of the different choices in each step of the proposed framework. We also demonstrated that the proposal surpasses the classification performance of other PUL (RC-SVM, PU-LP, and PE-PUC) or one-class learning (k-NN-based, k-Means-based, and Dense Autoencoder) algorithms in terms of F1. Considering the best results of any algorithm used in the experimental evaluation, PU-PUL can improve the classification performance from 2%, when using only 1 labeled document, to 28%, when 30 labeled documents are employed.

Multi-task analysis discriminative dictionary learning for one-class learning

One-class classification is a generalization of supervised learning based on one class of examples. It attracts growing attention in machine learning and data mining. In this paper, we propose a novel approach called multi-task dictionary learning for one-class learning (MTD-OC), which incorporates analysis discriminative dictionary learning into one-class learning. The analysis discriminative dictionary learning makes sure that dictionaries responding to different tasks are independent and discriminating as much as possible. The analysis discriminative dictionary learning simultaneously minimize l2,1-norm constraint, analysis incoherence term and sparse code extraction term, which aim to promote analysis incoherence and improve coding efficiency and accuracy for classification. The one-class classifier on the target task is then constructed by learning transfer knowledge from multiple source tasks. Here, one-class classification improves the performance of analysis discriminative dictionary, while analysis discriminative dictionary improves the performance of one-class classification term. In MTD-OC, the optimization function is formulated to deal with one-class classifier and analysis discriminative dictionary learning based on one class of examples. Then, we propose an iterative framework to solve the optimization function, and obtain the predictive classifier for the target class. Extensive experiments have shown that MTD-OC can improve the accuracy of one-class classifier by learning analysis discriminative dictionary from each task to construct a transfer classifier.

OCTID: a one-class learning-based Python package for tumor image detection.

Tumor tile selection is a necessary prerequisite in patch-based cancer whole slide image analysis, which is labor-intensive and requires expertise. Whole slides are annotated as tumor or tumor free, but tiles within a tumor slide are not. As all tiles within a tumor free slide are tumor free, these can be used to capture tumor-free patterns using the one-class learning strategy. We present a Python package, termed OCTID, which combines a pretrained convolutional neural network (CNN) model,Uniform Manifold Approximation and Projection (UMAP) and one-class support vector machine to achieve accurate tumor tile classification using a training set of tumor free tiles. Benchmarking experiments on four H&E image datasets achieved remarkable performance in terms of F1-score (0.90 ± 0.06), Matthews correlation coefficient (0.93 ± 0.05) and accuracy (0.94 ± 0.03). Detailed information can be found in the Supplementary File. Supplementary data are available at Bioinformatics online.

End-to-End Deep One-Class Learning for Anomaly Detection in UAV Video Stream.

In recent years, the use of drones for surveillance tasks has been on the rise worldwide. However, in the context of anomaly detection, only normal events are available for the learning process. Therefore, the implementation of a generative learning method in an unsupervised mode to solve this problem becomes fundamental. In this context, we propose a new end-to-end architecture capable of generating optical flow images from original UAV images and extracting compact spatio-temporal characteristics for anomaly detection purposes. It is designed with a custom loss function as a sum of three terms, the reconstruction loss (), the generation loss () and the compactness loss () to ensure an efficient classification of the “deep-one” class. In addition, we propose to minimize the effect of UAV motion in video processing by applying background subtraction on optical flow images. We tested our method on very complex datasets called the mini-drone video dataset, and obtained results surpassing existing techniques’ performances with an AUC of 85.3.

Continual Learning by Using Information of Each Class Holistically

Continual learning (CL) incrementally learns a sequence of tasks while solving the catastrophic forgetting (CF) problem. Existing methods mainly try to deal with CF directly. In this paper, we propose to avoid CF by considering the features of each class holistically rather than only the discriminative information for classifying the classes seen so far. This latter approach is prone to CF because the discriminative information for old classes may not be sufficiently discriminative for the new class to be learned. Consequently, in learning each new task, the network parameters for previous tasks have to be revised, which causes CF. With the holistic consideration, after adding new tasks, the system can still do well for previous tasks. The proposed technique is called Per-class Continual Learning (PCL). PCL has two key novelties. (1) It proposes a one-class learning based technique for CL, which considers features of each class holistically and represents a new approach to solving the CL problem. (2) It proposes a method to extract discriminative information after training to further improve the accuracy. Empirical evaluation shows that PCL markedly outperforms the state-of-the-art baselines for one or more classes per task. More tasks also result in more gains.

DCSNet: A Surface Defect Classification and Segmentation Model by One-Class Learning

Researches in surface defect classification and segmentation technology have been seen significant progress in recent years. However, there are few works on One-Class learning in this direction by a single model. In previous researches, some problems remain unsolved in the surface defect detection methods, e.g. the training needs a large number of samples and these models cannot classify and locate the surface defect accurately, etc. The main contribution in this work is that we summarize the overall ideas of previous research in network design and propose a multi-task model which could be trained only using a few of positive samples. Meanwhile, the experiments on AITEX detection datasets[1] which get 84.4% DR, 4.4% FAR and 34.2% MIOU, and conduct an ablation experiment in real industrial product dataset to validate the effect of different backbones on DCSNet. It’s worth mentioning that DCSNet provides a solution to the task of surface defect classification and segmentation based on One-Class learning. The code will be open source in ext-link-type="uri" xmlns:xlink="http://www.w3.org/1999/xlink" xlink:href="https://agit.ai/wyxxx/zhengtu">https://agit.ai/wyxxx/zhengtu.

A ubiquitous wheelchair fall detection system using low-cost embedded inertial sensors and unsupervised one-class SVM

Fall detector systems are one of the highly researched areas of Ambient Assisted Living (AAL) applications ensuring safety and autonomous living for the elderly. Today, despite the diversity of methods proposed for fall detectors, there is still a need to develop accurate and robust architectures, methods, and protocols for the detection of the occurrence of falls for a special-class of wheelchair-bound people. To address this issue, this paper proposes a wheelchair fall detector system based on a low-cost, light-weight inertial sensing method utilizing a hybrid scheme and unsupervised One-Class SVM (OCSVM) for detection of cases leading to a ‘fall’ anomaly during wheelchair maneuver and for the case of unassisted transfers. To make the system robust against noise, a novel hybrid multi-sensor fusion strategy combining Zero Angular Rate Update (ZART) and Complementary Filter (CF) to compensate sensor integral errors is utilized. A heterogeneous dataset is constructed using the publicly available Sis-Fall dataset to include possible fall cases due to unassisted transfers from wheelchairs and secondly, a prototype is developed to emulate the wheelchair system with the embedded inertial sensors to capture trends in the sensor measurements due to wheelchair tips and falls. The OCSVM anomaly detection technique is utilized to overcome the major disadvantage of supervised learning methods requiring huge datasets from activities performed by human subjects needed to guarantee higher accuracy rates from these detectors. In this regard, to capture the best features from the generated accelerometer and gyroscope feature set, the ReliefF algorithm is used. The proposed method is compared with the widely reported approaches in the literature for fall-detectors, i.e., threshold-based methods and other one-class learning approaches. It is demonstrated that the fall-detection accuracy (i.e., the g-mean score) was achieved up to 96% with the proposed method.

Robust One-Class Kernel Spectral Regression.

The kernel null-space technique is known to be an effective one-class classification (OCC) technique. Nevertheless, the applicability of this method is limited due to its susceptibility to possible training data corruption and the inability to rank training observations according to their conformity with the model. This article addresses these shortcomings by regularizing the solution of the null-space kernel Fisher methodology in the context of its regression-based formulation. In this respect, first, the effect of the Tikhonov regularization in the Hilbert space is analyzed, where the one-class learning problem in the presence of contamination in the training set is posed as a sensitivity analysis problem. Next, the effect of the sparsity of the solution is studied. For both alternative regularization schemes, iterative algorithms are proposed which recursively update label confidences. Through extensive experiments, the proposed methodology is found to enhance robustness against contamination in the training set compared with the baseline kernel null-space method, as well as other existing approaches in the OCC paradigm, while providing the functionality to rank training samples effectively.

One-Class Learning Towards Synthetic Voice Spoofing Detection

Human voices can be used to authenticate the identity of the speaker, but the automatic speaker verification (ASV) systems are vulnerable to voice spoofing attacks, such as impersonation, replay, text-to-speech, and voice conversion. Recently, researchers developed anti-spoofing techniques to improve the reliability of ASV systems against spoofing attacks. However, most methods encounter difficulties in detecting unknown attacks in practical use, which often have different statistical distributions from known attacks. Especially, the fast development of synthetic voice spoofing algorithms is generating increasingly powerful attacks, putting the ASV systems at risk of unseen attacks. In this work, we propose an anti-spoofing system to detect unknown synthetic voice spoofing attacks (i.e., text-to-speech or voice conversion) using one-class learning. The key idea is to compact the bona fide speech representation and inject an angular margin to separate the spoofing attacks in the embedding space. Without resorting to any data augmentation methods, our proposed system achieves an equal error rate (EER) of 2.19% on the evaluation set of ASVspoof 2019 Challenge logical access scenario, outperforming all existing single systems (i.e., those without model ensemble).

Minority manifold regularization by stacked auto-encoder for imbalanced learning

Imbalanced learning is considered one of the challenging problems in machine learning. This problem arises when a learning algorithm is biased toward the majority class due to the large proportion of the majority class data while detecting the minority class is of greater importance. In the present study, a novel method (MMRAE) is presented for imbalanced learning encompassing feature learning and classification steps. In the feature learning step, meaningful features are extracted from the minority data and their underlying manifold are captured by taking advantage of one-class learning approach through stacking two regularized auto-encoders. The existence of novel and different regularizers in each auto-encoder leads to a new representation with proper data discrimination which improves the between-class and within-class imbalanced problems. Then, in the classification step, the classification between the minority and majority class is performed by constructing a multilayer neural network using features learned throughout pre-training. The proposed method is extensively studied on six artificial and twenty real datasets in order to have a precise evaluation. Based on different criteria such as F-measure, G-mean, and AUC, the results represent considerable performance of the proposed method compared to several other existing methods.

From fault detection to one-class severity discrimination of 3D printers with one-class support vector machine

The lack of faulty condition data reduces the feasibility of supervised learning for fault detection or fault severity discrimination in new manufacturing technologies. To deal with this issue, one-class learning arises for building binary discriminative models using only healthy condition data. However, these models have not been extrapolated to severity discrimination. This paper proposes to extend OCSVM, which is typically used for fault detection, to 3D printer fault severity discrimination. First, a set of features is extracted from a set of normal signals. An optimized OCSVM model is obtained by tuning the kernel and model hyperparameters. The resulting models are evaluated for fault detection and fault severity discrimination using a proposed performance evaluation approach. Experimental comparisons for belt-based faults in 3D printers show that the distance to the hyperplane has the information to discriminate the severity level, and its use is feasible. The proposed hyperparameter optimization technique improves the OCSVM for fault detection and severity discrimination compared to some other methods.

Catering for unique tastes: Targeting grey-sheep users recommender systems through one-class machine learning

In recommendation systems, the grey-sheep problem refers to users with unique preferences and tastes that make it difficult to develop accurate profiles. That is, the similarity search approach typically followed during the recommendation process fails to yield good results. Most research does not focus on such users and thus fails to cater to more exotic tastes and emerging trends, leading to a subsequent loss in revenue and marketing opportunities. One suggested solution is to use one-class classification to generate a prediction list for these users, where decision boundaries are learned that distinguish between normal and grey-sheep users. In this paper, we present the grey-sheep one-class recommendation (GSOR) framework designed to create accurate prediction models while taking both regular and grey-sheep users into account. In addition, we introduce a novel grey-sheep movie recommendation benchmark to be used by current and future researchers. When evaluating our GSOR framework against this benchmark, our results indicate the value of combining cluster analysis, outlier detection, and one-class learning to generate relevant and timely recommendation lists from data sets that contain grey-sheep users. Specifically, by employing one-class decision tree algorithms, our GSOR framework was able to outperform traditional collaborative filtering-based recommendation systems in both accuracy and model construction time. Furthermore, we report that having grey-sheep users in the system often had a positive impact on the learning and recommendation processes.

Transfer learning-based one-class dictionary learning for recommendation data stream

Recommendation system has attracted growing attention in machine learning and data mining. In addition, one-class learning has been proposed to build a predictive classifier mainly based on the preferred data, and has been verified to be an efficient method for recommendation system. In this paper, we propose a novel recommendation model called transfer learning-based one-class dictionary learning for recommendation data stream (DOC-SVM), which incorporates transfer learning and discriminative dictionary learning into one-class data stream learning. The proposed method can reduce the effect of the uncertain data on the classifier and transfer knowledge from the recent historical chunk to the current chunk to construct a predictive classifier for the current chunk. We then propose an iterative framework to solve the optimization function, and obtain the predictive classifier for the current chunk. Extensive experiments has been conducted and the results show that the proposed method can deliver better performance than the state-of-the-art methods.

Fast and Memory-Efficient Import Vector Domain Description

One-class learning is a classical and hard computational intelligence task. In the literature, there are some effective and powerful solutions to address the problem. There are examples in the kernel machines realm, Support Vector Domain Description, and the recently proposed Import Vector Domain Description (IVDD), which directly delivers the sample probability of belonging to the class. Here, we propose and discuss two optimization techniques for IVDD to significantly improve the memory footprint and consequently to scale to datasets that are larger than the original formulation. We propose two strategies. First, we propose using random features to approximate the gaussian kernel together with a primal optimization algorithm. Second, we propose a Nystrom-like approximation of the functional together with a fast converging and accurate self-consistent algorithm. In particular, we replace the a posteriori sparsity of the original optimization method of IVDD by randomly selecting a priori landmark samples in the dataset. We find this second approximation to be superior. Compared to the original IVDD with the RBF kernel, it achieves high accuracy, is much faster, and grants huge memory savings.

Video anomaly detection and localization using motion-field shape description and homogeneity testing

Detection and localization of abnormal behaviors in surveillance videos of crowded scenes is challenging, where high-density people and various objects performing highly unpredictable motions lead to severe occlusions, making object segmentation and tracking extremely difficult. We associate the optical flows between multiple frames to capture short-term trajectories and introduce the histogram-based shape descriptor to describe such short-term trajectories, which reflects faithfully the motion trend and details in local patches. Furthermore, we propose a method to detect anomalies over time and space by judging whether the similarities between the testing sample and the retrieved K-NN samples follow the pattern distribution of homogeneous intra-class similarities, which is unsupervised one-class learning requiring no clustering nor prior assumption. Such a scheme can adapt to the whole scene, since the probability is used to judge and the calculation of probability is not affected by motion distortions arising from perspective distortion, which gains advantage over the existing solutions. We conduct experiments on real-world surveillance videos, and the results demonstrate that the proposed method can reliably detect and locate the abnormal events in video sequences, outperforming the state-of-the-art approaches.

Pattern-Growth Based Mining Mouse-Interaction Behavior for an Active User Authentication System

Analyzing mouse-interaction behaviors for implicitly identifying computer users has received growing interest from security and biometric researchers. This study presents a simple but efficient active user authentication system by modeling mouse-interaction behavior, which is accurate and competent for future deployments. A pattern-growth-based mining method is proposed to extract frequent behavior segments, in obtaining a stable and discriminative representation of mouse-interaction behavior. Then procedural features are extracted to provide an accurate and fine-grained characterization of the behavior segments. A SVM-based decision procedure using one-class learning techniques is applied to the feature space for performing authentication. Analyses are conducted using data from around 1,526,400 mouse operations of 159 participants, and the authentication performance is evaluated across various application scenarios and tasks. Our experimental results show that characteristics from frequent behavior segments are more stable and discriminative than those from holistic behavior, and the system achieves a practically useful level of performance with FAR of 0.09 percent and FRR of 1 percent. Additional experiments on usability to sample length, reliability to application task, scalability to user size, robustness to mimic attack, and response to behavior change are provided to further explore the applicability. We also compare the proposed approach with the state-of-the-art approaches for the collected data.