Semi-supervised Outlier Detection Research Articles

Dual-MGAN: An Efficient Approach for Semi-supervised Outlier Detection with Few Identified Anomalies

Outlier detection is an important task in data mining, and many technologies for it have been explored in various applications. However, owing to the default assumption that outliers are not concentrated, unsupervised outlier detection may not correctly identify group anomalies with higher levels of density. Although high detection rates and optimal parameters can usually be achieved by using supervised outlier detection, obtaining a sufficient number of correct labels is a time-consuming task. To solve these problems, we focus on semi-supervised outlier detection with few identified anomalies and a large amount of unlabeled data. The task of semi-supervised outlier detection is first decomposed into the detection of discrete anomalies and that of partially identified group anomalies, and a distribution construction sub-module and a data augmentation sub-module are then proposed to identify them, respectively. In this way, the dual multiple generative adversarial networks (Dual-MGAN) that combine the two sub-modules can identify discrete as well as partially identified group anomalies. In addition, in view of the difficulty of determining the stop node of training, two evaluation indicators are introduced to evaluate the training status of the sub-GANs. Extensive experiments on synthetic and real-world data show that the proposed Dual-MGAN can significantly improve the accuracy of outlier detection, and the proposed evaluation indicators can reflect the training status of the sub-GANs.

An independent central point OPTICS clustering algorithm for semi-supervised outlier detection of continuous glucose measurements

Continuous glucose monitoring (CGM) collects a host of time-series sensor data and is committed to fully automated systems for glucose control as an essential part. Outlier data in CGM measurements caused by faults may seriously affect the computation of insulin infusion rates and endanger the safety of patients. In this paper, a semi-supervised outlier detection method is proposed for anomaly detection of glucose concentration measurements based on a density-based clustering algorithm, named independent central point OPTICS (ICP-OPTICS). An optimization function is designed to improve the clustering performance and solve the problem of outlier detection with the distance measurement and information entropy of weighted time series. The proposed method in the application can be configured automatically with no prior knowledge except only some clean samples. The UVa/Padova simulator and real dataset are used to evaluate the performance of the method. Compared with the present work, the statistical results show that the method is much better in effectiveness and superiority. Furthermore, the proposed method also provides a possible reference significance for outlier detection in other practice applications.

Water Meter Replacement Recommendation for Municipal Water Distribution Networks using Ensemble Outlier Detection Methods

Due to their structure and usage condition, water meters face degradation, breaking, freezing, and leakage problems. There are various studies intended to determine the appropriate time to replace degraded ones. Earlier studies have used several features, such as user meteorological parameters, usage conditions, water network pressure, and structure of meters to detect failed water meters. This article proposes a recommendation framework that uses registered water consumption values as input data and provides meter replacement recommendations. This framework takes time series of registered consumption values and preprocesses them in two rounds to extract effective features. Then, multiple un-/semi-supervised outlier detection methods are applied to the processed data and assigns outlier/normal labels to them. At the final stage, a hypergraph-based ensemble method receives the labels and combines them to discover the suitable label. Due to the unavailability of ground truth labeled data for meter replacement, we compare our method with respect to its FPR and two internal metrics: Dunn index and Davies-Bouldin Index. Results of our comparative experiments show that the proposed framework detects more compact clusters with smaller variance.

A Graph-Based Method for Active Outlier Detection With Limited Expert Feedback

Labeled data, particularly for the outlier class, are difficult to obtain. Thus, outlier detection is typically regarded as an unsupervised learning problem. However, it still has an opportunity to obtain few labeled data. For example, a human analyst can give feedback to few data when he/she examines the results of an unsupervised outlier detection method. Moreover, the widely used unsupervised method for outlier detection cannot only take the labeled data into consideration nor use them properly. In this study, we first propose a graph-based method to endow the unsupervised method with the ability to consider few labeled data. Then, we extend our semi-supervised method to active outlier detection by incorporating the query strategy that selects top-ranked outliers. Comprehensive experiments on 12 real-world datasets demonstrate that our semi-supervised outlier detection method is comparable with the best of state-of-the-art approaches, and our active outlier detection method outperforms state-of-the-art methods.

An Improved Semisupervised Outlier Detection Algorithm Based on Adaptive Feature Weighted Clustering

There exist already various approaches to outlier detection, in which semisupervised methods achieve encouraging superiority due to the introduction of prior knowledge. In this paper, an adaptive feature weighted clustering-based semisupervised outlier detection strategy is proposed. This method maximizes the membership degree of a labeled normal object to the cluster it belongs to and minimizes the membership degrees of a labeled outlier to all clusters. In consideration of distinct significance of features or components in a dataset in determining an object being an inlier or outlier, each feature is adaptively assigned different weights according to the deviation degrees between this feature of all objects and that of a certain cluster prototype. A series of experiments on a synthetic dataset and several real-world datasets are implemented to verify the effectiveness and efficiency of the proposal.

Semi-Supervised Outlier Detection with Only Positive and Unlabeled Data Based on Fuzzy Clustering

The task of semi-supervised outlier detection is to find the instances that are exceptional from other data, using some labeled examples. In many applications such as fraud detection and intrusion detection, this issue becomes more important. Most existing techniques are unsupervised. On the other hand, semi-supervised approaches use both negative and positive instances to detect outliers. However, in many real world applications, very few positive labeled examples are available. This paper proposes an innovative approach to address this problem. The proposed method works as follows. First, some reliable negative instances are extracted by a kNN-based algorithm. Afterwards, fuzzy clustering using both negative and positive examples is utilized to detect outliers. Experimental results on real data sets demonstrate that the proposed approach outperforms the previous unsupervised state-of-the-art methods in detecting outliers.

Semi-supervised outlier detection based on fuzzy rough C-means clustering

This paper presents a fuzzy rough semi-supervised outlier detection (FRSSOD) approach with the help of some labeled samples and fuzzy rough C-means clustering. This method introduces an objective function, which minimizes the sum squared error of clustering results and the deviation from known labeled examples as well as the number of outliers. Each cluster is represented by a center, a crisp lower approximation and a fuzzy boundary by using fuzzy rough C-means clustering and only those points located in boundary can be further discussed the possibility to be reassigned as outliers. As a result, this method can obtain better clustering results for normal points and better accuracy for outlier detection. Experiment results show that the proposed method, on average, keep, or improve the detection precision and reduce false alarm rate as well as reduce the number of candidate outliers to be discussed.