Unsupervised Outlier Detection Research Articles

A Temporal Dependency Based Multi-modal Active Learning Approach for Audiovisual Event Detection

In this work, two novel active learning approaches for the annotation and detection of audiovisual events are proposed. The assumption behind the proposed approaches is that events are susceptible to substantively deviate from the distribution of normal observations and therefore should be lying in regions of low density. Thus, it is believed that an event detection model can be trained more efficiently by focusing on samples that appear to be inconsistent with the majority of the dataset. The first approach is an uni-modal method which consists in using rank aggregation to select informative samples which have previously been ranked using different unsupervised outlier detection techniques in combination with an uncertainty sampling technique. The information used for the sample selection stems from an unique modality (e.g. video channel). Since most active learning approaches focus on one target channel to perform the selection of informative samples and thus do not take advantage of potentially useful and complementary information among correlated modalities, we propose an extension of the previous uni-modal approach to multi-modality. From a target pool of instances belonging to a specific modality, the uni-modal approach is used to select and manually label a set of informative instances. Additionally, a second set of automatically labelled instances of the target pool is generated, based on a transfer of information stemming from an auxiliary modality which is temporally dependent to the target one. Both sets of labelled instances (automatically and manually labelled instances) are used for the semi-supervised training of a classification model to be used in the next active learning iteration. Both methods have been assessed on a set of participants selected from the UUlmMAC dataset and have proven to be effective in substantially reducing the cost of manual annotation required for the training of a facial event detection model. The assessment is done based on two different methods: Support Vector Data Description and expected similarity estimation. Furthermore, given an appropriate sampling approach, the multi-modal approach outperforms its uni-modal counterpart in most of the cases.

Unsupervised outlier detection for time series by entropy and dynamic time warping

In the last decade, outlier detection for temporal data has received much attention from data mining and machine learning communities. While other works have addressed this problem by two-way approaches (similarity and clustering), we propose in this paper an embedded technique dealing with both methods simultaneously. We reformulate the task of outlier detection as a weighted clustering problem based on entropy and dynamic time warping for time series. The outliers are then detected by an optimization problem of a new proposed cost function adapted to this kind of data. Finally, we provide some experimental results for validating our proposal and comparing it with other methods of detection.

Unsupervised Sequential Outlier Detection With Deep Architectures.

Unsupervised outlier detection is a vital task and has high impact on a wide variety of applications domains, such as image analysis and video surveillance. It also gains long-standing attentions and has been extensively studied in multiple research areas. Detecting and taking action on outliers as quickly as possible are imperative in order to protect network and related stakeholders or to maintain the reliability of critical systems. However, outlier detection is difficult due to the one class nature and challenges in feature construction. Sequential anomaly detection is even harder with more challenges from temporal correlation in data, as well as the presence of noise and high dimensionality. In this paper, we introduce a novel deep structured framework to solve the challenging sequential outlier detection problem. We use autoencoder models to capture the intrinsic difference between outliers and normal instances and integrate the models to recurrent neural networks that allow the learning to make use of previous context as well as make the learners more robust to warp along the time axis. Furthermore, we propose to use a layerwise training procedure, which significantly simplifies the training procedure and hence helps achieve efficient and scalable training. In addition, we investigate a fine-tuning step to update all parameters set by incorporating the temporal correlation in the sequence. We further apply our proposed models to conduct systematic experiments on five real-world benchmark data sets. Experimental results demonstrate the effectiveness of our model, compared with other state-of-the-art approaches.

Bagged Subspaces for Unsupervised Outlier Detection

In many domains, important events are not represented as the common scenario, but as deviations from the rule. The importance and impact associated with these particular, outnumbered, deviant, and sometimes even previously unseen events is directly related to the application domain (e.g., breast cancer detection, satellite image classification, etc.). The detection of these rare events or outliers has recently been gaining popularity as evidenced by the wide variety of algorithms currently available. These algorithms are based on different assumptions about what constitutes an outlier, a characteristic pointing toward their integration in an ensemble to improve their individual detection rate. However, there are two factors that limit the use of current ensemble outlier detection approaches: first, in most cases, outliers are not detectable in full dimensionality, but instead are located in specific subspaces of data; and second, despite the expected improvement on detection rate achieved using an ensemble of detectors, the computational efficiency of the ensemble will increase linearly as the number of components increases. In this article, we propose an ensemble approach that identifies outliers based on different subsets of features and subsamples of data, providing more robust results while improving the computational efficiency of similar ensemble outlier detection approaches.

Biometric template extraction from a heartbeat signal captured from fingers

Authentication of remote users is very important for security and privacy of multimedia content that is often accessed online. Heartbeat signal has emerged as biometric modality suitable for remote authentication for its privacy and liveness property. In order to improve acceptability of this modality, we propose a method of biometric template extraction from a heartbeat signal captured from fingers. An unsupervised outlier detection method is employed to select the most regular heartbeats from a specimen. In order to mitigate the effect of heart rate variability (HRV), morphology of the selected heartbeat is aligned by a piecewise-uniform method. Then, a template is formed by averaging aligned heartbeats and presented in a lower dimensional space by the principal component analysis. Authentication performance of the template was evaluated using a database collected from 112 individuals in multiple sessions by a handheld ECG device. We also implemented four state-of-the-art templates and tested them by the same database. Experimental results indicate that authentication performance of the proposed template is superior to those of any of these templates. It is also suitable for remote authentication using mobile computing devices for its computational efficiency and compactness.

An Efficient Unsupervised Clustered Adaptive Antihub Technique for Outlier Detection in High Dimensional Data

Objective: The objective of this paper is to find the inconsistent objects in data which has high dimension through reduced computation time and increased accuracy. Methods: Hubness specifically Antihubs (points that rarely occur in k nearest neighbor lists) is the newly recognized concept for handling data which has high dimension. The advanced version of Antihub is Antihub2 which is for reconsidering the outlier score of a point obtained by the Antihub method. However, regarding computation time, Antihub2 runs slower. This paper institutes an approach called AdaptiveAntihub2Clust, which is a clustered Adaptive Antihub technique for unsupervised outlier detection to reduce computation time and to improve the accuracy. Findings: The results of an existing Antihub2 method is compared with the proposed method called AdaptiveAntihub2Clust. The experimental results elucidate that AdaptiveAntihub2Clust outperforms well than Antihub2 and also resolved that there is not only a substantial decrease in computation time but also progress in accuracy occurred while the newly built approach is practically used for finding outliers. Applications: The irrelevant objects may ascend due to numerous faults. Detection of such objects identifies the mistakes and fraud before they deteriorate with terrible significances and cleanses the data for further processing.

Outlier Detection in High Dimension Data Based On Multimodality And Neighbourhood Size Using KNN Method

Outlier detection in high-dimensional data presents various challenges resulting from the “curse of dimensionality.” A prevailing view is that distance concentration, i.e., the tendency of distances in high-dimensional data to become indiscernible, hinders the detection of outliers by making distance-based methods label all points as almost equally good outliers. In this paper, we provide evidence supporting the opinion that such a view is too simple, by demonstrating that distance-based methods can produce more contrasting outlier scores in high-dimensional settings. Furthermore, we show that high dimensionality can have a different impact, by reexamining the notion of reverse nearest neighbors in the unsupervised outlier-detection context. Namely, it was recently observed that the distribution of points’ reverse-neighbor counts becomes skewed in high dimensions, resulting in the phenomenon known as hubness. We provide insight into how some points (antihubs) appear very infrequently in k-NN lists of other points, and explain the connection between antihubs, outliers, and existing unsupervised outlier-detection methods. By evaluating the classic k-NN method, the angle-based technique designed for high-dimensional data, the density-based local outlier factor and influenced outlierness methods, and antihub-based methods on various synthetic and real-world data sets, we offer novel insight into the usefulness of reverse neighbor counts in unsupervised outlier detection.

New framework for unsupervised universal steganalysis via SRISP-aided outlier detection

Formulating steganalysis as a binary classification problem has been highly successful. However, the existing detection algorithms are difficult to obtain high detection accuracy when applied in real-world circumstances. Because so-called model mismatch problem often occurs owing to unknown cover source and embedding parameters. To avoid the mess of model mismatch, we propose a new unsupervised universal steganalysis framework to detect individual stego images. First, cover images with statistical properties similar to those of the given test image are searched from a retrieval cover database to establish an aided cover sample set. Second, unsupervised outlier detection is performed on a test set composed of the given test image and its aided cover sample set to determine the type (cover or stego) of the given test image. Our proposed framework, called Similarity Retrieval of Image Statistical Properties (SRISP)-aided unsupervised outlier detection, requires no training, and thus it does not suffer from model mismatch. The framework employs standard steganalysis features and detects each test image individually. Experimental results illustrate that the framework substantially outperforms one-class support vector machine and the traditional unsupervised outlier detectors without considering SRISP; its detection performance is independent of the proportion of stego images in the test samples.

A Line Rate Outlier Filtering FPGA NIC using 10GbE Interface

As data sets grow rapidly in size and the number, an outlier detection that filters unnecessary normal information becomes important. In this paper, we propose to move the unsupervised outlier detection from an application layer to a network interface card (NIC). Only anomalous items or events are received for a network protocol stack and the other packets are discarded at the NIC. The demands for storage and computation costs at a host are thus dramatically reduced. However, because normal items are discarded at the NIC and the application layer can no longer know what is normal, in our approach, the application at the host periodically peeks at the NIC buffer. We select an outlier detection based on the Mahalanobis distance as one of the simplest algorithms. Our approach is implemented on an FPGA-based NIC that has 10GbE interfaces. The sampling frequency of the NIC buffer vs. outlier detection precision is analyzed. Real experiments using the FPGA NIC demonstrate a 14,000,000 samples-per-second throughput in performance, which is close to the 10GbE line rate.

On the evaluation of unsupervised outlier detection: measures, datasets, and an empirical study

The evaluation of unsupervised outlier detection algorithms is a constant challenge in data mining research. Little is known regarding the strengths and weaknesses of different standard outlier detection models, and the impact of parameter choices for these algorithms. The scarcity of appropriate benchmark datasets with ground truth annotation is a significant impediment to the evaluation of outlier methods. Even when labeled datasets are available, their suitability for the outlier detection task is typically unknown. Furthermore, the biases of commonly-used evaluation measures are not fully understood. It is thus difficult to ascertain the extent to which newly-proposed outlier detection methods improve over established methods. In this paper, we perform an extensive experimental study on the performance of a representative set of standard k nearest neighborhood-based methods for unsupervised outlier detection, across a wide variety of datasets prepared for this purpose. Based on the overall performance of the outlier detection methods, we provide a characterization of the datasets themselves, and discuss their suitability as outlier detection benchmark sets. We also examine the most commonly-used measures for comparing the performance of different methods, and suggest adaptations that are more suitable for the evaluation of outlier detection results.

Hubness in Unsupervised Outlier Detection Techniques for High Dimensional Data –A Survey

Hubness in Unsupervised Outlier Detection Techniques for High Dimensional Data –A Survey

A decomposition of the outlier detection problem into a set of supervised learning problems

Outlier detection methods automatically identify instances that deviate from the majority of the data. In this paper, we propose a novel approach for unsupervised outlier detection, which re-formulates the outlier detection problem in numerical data as a set of supervised regression learning problems. For each attribute, we learn a predictive model which predicts the values of that attribute from the values of all other attributes, and compute the deviations between the predictions and the actual values. From those deviations, we derive both a weight for each attribute, and a final outlier score using those weights. The weights help separating the relevant attributes from the irrelevant ones, and thus make the approach well suitable for discovering outliers otherwise masked in high-dimensional data. An empirical evaluation shows that our approach outperforms existing algorithms, and is particularly robust in datasets with many irrelevant attributes. Furthermore, we show that if a symbolic machine learning method is used to solve the individual learning problems, the approach is also capable of generating concise explanations for the detected outliers.

Reverse Nearest Neighbors in Unsupervised Distance-Based Outlier Detection

Outlier detection in high-dimensional data presents various challenges resulting from the “curse of dimensionality.” A prevailing view is that distance concentration, i.e., the tendency of distances in high-dimensional data to become indiscernible, hinders the detection of outliers by making distance-based methods label all points as almost equally good outliers. In this paper, we provide evidence supporting the opinion that such a view is too simple, by demonstrating that distance-based methods can produce more contrasting outlier scores in high-dimensional settings. Furthermore, we show that high dimensionality can have a different impact, by reexamining the notion of reverse nearest neighbors in the unsupervised outlier-detection context. Namely, it was recently observed that the distribution of points' reverse-neighbor counts becomes skewed in high dimensions, resulting in the phenomenon known as hubness. We provide insight into how some points (antihubs) appear very infrequently in k-NN lists of other points, and explain the connection between antihubs, outliers, and existing unsupervised outlier-detection methods. By evaluating the classic k-NN method, the angle-based technique designed for high-dimensional data, the density-based local outlier factor and influenced outlierness methods, and antihub-based methods on various synthetic and real-world data sets, we offer novel insight into the usefulness of reverse neighbor counts in unsupervised outlier detection.

English

2 ABSTRACT: Outlier detection is the process of finding outlying pattern from a given dataset. Outlier detection became important subject in different knowledge domains. Data size is getting doubled every years there is a need to detect outliers in large datasets as early as possible. In high-dimensional data outlier detection presents various challenges because of curse of dimensionality. By examining again the notion of reverse nearest neighbors in the unsupervised outlier-detection context, high dimensionality can have a different impact. In high dimensions it was observed that the distribution of points in reverse-neighbor counts becomes skewed .This proposed work aims at developing and comparing some of the unsupervised outlier detection methods and propose a way to improve them. This proposed work goes in details about the development and analysis of outlier detection algorithms such as Local Outlier Factor(LOF), Local Distance-Based Outlier Factor(LDOF) , Influenced Outliers and .The concepts of these methods are then combined to implement a new method with distributed approach which improves the results of the previous mentioned ones with reference to speed, complexity and accuracy.

Ensembles for unsupervised outlier detection

Ensembles for unsupervised outlier detection is an emerging topic that has been neglected for a surprisingly long time (although there are reasons why this is more difficult than supervised ensembles or even clustering ensembles). Aggarwal recently discussed algorithmic patterns of outlier detection ensembles, identified traces of the idea in the literature, and remarked on potential as well as unlikely avenues for future transfer of concepts from supervised ensembles. Complementary to his points, here we focus on the core ingredients for building an outlier ensemble, discuss the first steps taken in the literature, and identify challenges for future research.

Unsupervised Outlier Detection in Streaming Data Using Weighted Clustering

Unsupervised Outlier Detection in Streaming Data Using Weighted Clustering

A survey on unsupervised outlier detection in high‐dimensional numerical data

AbstractHigh‐dimensional data in Euclidean space pose special challenges to data mining algorithms. These challenges are often indiscriminately subsumed under the term ‘curse of dimensionality’, more concrete aspects being the so‐called ‘distance concentration effect’, the presence of irrelevant attributes concealing relevant information, or simply efficiency issues. In about just the last few years, the task of unsupervised outlier detection has found new specialized solutions for tackling high‐dimensional data in Euclidean space. These approaches fall under mainly two categories, namely considering or not considering subspaces (subsets of attributes) for the definition of outliers. The former are specifically addressing the presence of irrelevant attributes, the latter do consider the presence of irrelevant attributes implicitly at best but are more concerned with general issues of efficiency and effectiveness. Nevertheless, both types of specialized outlier detection algorithms tackle challenges specific to high‐dimensional data. In this survey article, we discuss some important aspects of the ‘curse of dimensionality’ in detail and survey specialized algorithms for outlier detection from both categories. © 2012 Wiley Periodicals, Inc. Statistical Analysis and Data Mining, 2012

In-network outlier detection in wireless sensor networks

To address the problem of unsupervised outlier detection in wireless sensor networks, we develop an approach that (1) is flexible with respect to the outlier definition, (2) computes the result in-network to reduce both bandwidth and energy usage,(3) only uses single hop communication thus permitting very simple node failure detection and message reliability assurance mechanisms (e.g., carrier-sense), and (4) seamlessly accommodates dynamic updates to data. We examine performance using simulation with real sensor data streams. Our results demonstrate that our approach is accurate and imposes a reasonable communication load and level of power consumption.

Outlier One Class Support Vector Machines

One-Class Support Vector Machines (OCSVMs) distinguish outliers by computing a hyper-plane in feature space. The choice of the origin as separation point is arbitrary, which affects the decision boundary, and the distribution of samples has impact on the performance. Expanding the algorithm into solving two-class classification problems overcomes the drawbacks to a certain degree. However, the class imbalance problem is serious and the labeled outliers are rare or even non-existing. In this paper, a new Outlier OCSVM is proposed and a framework is designed for unsupervised outlier detection. Respectively scored by distance from hyper-plane and probabilistic output value, two definitions of outlier degree are presented. After picking out some suspicious outliers via combining the two criterions of outlier degree, the adjusted Outlier OCSVM starts the training operations, two parts of the dataset are updated interactively through comparison of the outputs. Experiment results on benchmark datasets show that the method can effectively improve the detection rate and reduce false positive rate, easy and reliable.

On-Line Unsupervised Outlier Detection Using Finite Mixtures with Discounting Learning Algorithms

Outlier detection is a fundamental issue in data mining, specifically in fraud detection, network intrusion detection, network monitoring, etc. SmartSifter is an outlier detection engine addressing this problem from the viewpoint of statistical learning theory. This paper provides a theoretical basis for SmartSifter and empirically demonstrates its effectiveness. SmartSifter detects outliers in an on-line process through the on-line unsupervised learning of a probabilistic model (using a finite mixture model) of the information source. Each time a datum is input SmartSifter employs an on-line discounting learning algorithm to learn the probabilistic model. A score is given to the datum based on the learned model with a high score indicating a high possibility of being a statistical outlier. The novel features of SmartSifter are: (1) it is adaptive to non-stationary sources of datas (2) a score has a clear statistical/information-theoretic meanings (3) it is computationally inexpensives and (4) it can handle both categorical and continuous variables. An experimental application to network intrusion detection shows that SmartSifter was able to identify data with high scores that corresponded to attacks, with low computational costs. Further experimental application has identified a number of meaningful rare cases in actual health insurance pathology data from Australia's Health Insurance Commission.