Outlier Degree Research Articles

Anomaly Detection of Store Cash Register Data Based on Improved LOF Algorithm

As the cash register system gradually prevailed in shopping malls, detecting the abnormal status of the cash register system has gradually become a hotspot issue. This paper analyzes the transaction data of a shopping mall. When calculating the degree of data difference, the coefficient of variation is used as the attribute weight; the weighted Euclidean distance is used to calculate the degree of difference; and k-means clustering is used to classify different time periods. It applies the LOF algorithm to detect the outlier degree of transaction data at each time period, sets the initial threshold to detect outliers, deletes the outliers, and then performs SAX detection on the data set. If it does not pass the test, then it will gradually expand the outlying domain and repeat the above process to optimize the outlier threshold to improve the sensitivity of detection algorithm and reduce false positives.

Two-View Orthographic Epipolar Geometry: Minimal and Optimal Solvers

We will in this paper present methods and algorithms for estimating two-view geometry based on an orthographic camera model. We use a previously neglected nonlinear criterion on rigidity to estimate the calibrated essential matrix. We give efficient algorithms for estimating it minimally (using only three point correspondences), in a least squares sense (using four or more point correspondences), and optimally with respect to the number of inliers. The inlier-optimal algorithm is based on a three-point solver and gives a fourth-order polynomial time algorithm. These methods can be used as building blocks to robustly find inlier correspondences in the presence of high degrees of outliers. We show experimentally that our methods can be used in many instances, where the orthographic camera model isn’t generally used. A case of special interest is situations with repetitive structures, which give high amounts of outliers in the initial feature point matching.

Outlier detection based on multi-dimensional clustering and local density

Outlier detection is an important task in data mining. In fact, it is difficult to find the clustering centers in some sophisticated multidimensional datasets and to measure the deviation degree of each potential outlier. In this work, an effective outlier detection method based on multi-dimensional clustering and local density (ODBMCLD) is proposed. ODBMCLD firstly identifies the center objects by the local density peak of data objects, and clusters the whole dataset based on the center objects. Then, outlier objects belonging to different clusters will be marked as candidates of abnormal data. Finally, the top N points among these abnormal candidates are chosen as final anomaly objects with high outlier factors. The feasibility and effectiveness of the method are verified by experiments.

Probability Distribution and Deviation Information Fusion Driven Support Vector Regression Model and Its Application

In modeling, only information from the deviation between the output of the support vector regression (SVR) model and the training sample is considered, whereas the other prior information of the training sample, such as probability distribution information, is ignored. Probabilistic distribution information describes the overall distribution of sample data in a training sample that contains different degrees of noise and potential outliers, as well as helping develop a high-accuracy model. To mine and use the probability distribution information of a training sample, a new support vector regression model that incorporates probability distribution information weight SVR (PDISVR) is proposed. In the PDISVR model, the probability distribution of each sample is considered as the weight and is then introduced into the error coefficient and slack variables of SVR. Thus, the deviation and probability distribution information of the training sample are both used in the PDISVR model to eliminate the influence of noise and outliers in the training sample and to improve predictive performance. Furthermore, examples with different degrees of noise were employed to demonstrate the performance of PDISVR, which was then compared with those of three SVR-based methods. The results showed that PDISVR performs better than the three other methods.

K-NN Based Outlier Detection Technique on Intrusion Dataset

Outliers in the database are the objects that deviate from the rest of the dataset by some measure. The Nearest Neighbor Outlier Factor is considering to measure the degree of outlier-ness of the object in the dataset. Unlike the other methods like Local Outlier Factor, this approach shows the interest of a point from both neighbors and reverse neighbors, and after that, an object comes into consideration. We have observed that in GBBK algorithm that based on K-NN, used quick sort to find k nearest neighbors that take O (N log N) time. However, in proposed method, the time required for searching on K times which complete in O (KN) time to find k nearest neighbors (k < < log N). As a result, the proposed method improves the time complexity. The NSL-KDD and Fisher iris dataset is used, and experimental results compared with the GBBK method. The result is same in both the methods, but the proposed method takes less time for computation.

Research on the Intrusion Detection Method Based on Differentiated Cluster Center Offset Measure

In this paper, a kind of local outlier mining method based on differentiated cluster center offset measure is proposed through which the outlier degree of sample can be calculated by use of the normal behavior model constructed by normal data sample and the preset anomaly threshold value, and whether the testing sample belong to intrusion behavior can thus be determined. Furthermore, KDD99 data set is also utilized to test the said method, and the experimental results show that the method proposed in this paper possesses higher detection rate and lower false alarm rate.

Graph clustering using k-Neighbourhood Attribute Structural similarity

A simple and novel approach to identify the clusters based on structural and attribute similarity in graph network is proposed which is a fundamental task in community detection. We identify the dense nodes using Local Outlier Factor (LOF) approach that measures the degree of outlierness, forms a basic intuition for generating the initial core nodes for the clusters. Structural Similarity is identified using k-neighbourhood and Attribute similarity is estimated through Similarity Score among the nodes in the group of structural clusters. An objective function is defined to have quick convergence in the proposed algorithm. Through extensive experiments on dataset (DBLP) with varying sizes, we demonstrate the effectiveness and efficiency of our proposed algorithm k-Neighbourhood Attribute Structural (kNAS) over state-of-the-art methods which attempt to partition the graph based on structural and attribute similarity in field of community detection. Additionally, we find the qualitative and quantitative benefit of combining both the similarities in graph.

On the Development of an Exponentiated F Test for One-way ANOVA in the Presence of Outlier(s)

The classical Fisher-Snedecor test which compares several population means depends on the underlined assumptions which include; independent of populations, constant variance and absence of outlier among others .Arguably the source of violation of some of these assumptions is the outlier which lead to unequal variances. Outlier leads to inequality in the variances of the populations which consequently leads to the failure of the classical-F to take correct decision in terms of the null hypothesis. A series of robust tests have been carried out to ameliorate these lapses with some degrees of inaccuracies and limitations in terms of inflating the type 1 error and the power of different combination of parameters at various sample sizes while still uses the conventional F-table. This study focuses on developing robust F-test called exponentiated F test with the introduction of one shape parameter to the conventional F-distribution capable of taking decisions on ANOVA that are robust to the existence of outlier. The performance of the robust F test was compared with the existing F-tests in the literature using the power of test. Real life and simulated data were used to illustrate the applicability and efficiency of the proposed distribution over the existing ones. Experimental data with balanced and unbalanced design were used with populations sizes k=3 and k=5 were simulated with 10000 replications and varying degrees of outliers were ejected randomly. The results obtained indicate that the Proposed Exponentiated-F test is uniformly most powerful than the conventional-F tests for analysis of variance in the presence of outlier and is therefore recommended for use by researchers.

A relevant subspace based contextual outlier mining algorithm

For high-dimensional and massive data sets, a relevant subspace based contextual outlier detection algorithm is proposed. Firstly, the relevant subspace, which can effectively describe the local distribution of the various data sets, is redefined by using local sparseness of attribute dimensions. Secondly, a local outlier factor calculation formula in the relevant subspace is defined with probability density of local data sets, and the formula can effectively reflect the outlier degree of data object that does not obey the distribution of the local data set in the relevant subspace. Thirdly, attribute dimensions of constituting the relevant subspace and local outlier factor are defined as the contextual information, which can improve the interpretability and comprehensibility of outlier. Fourthly, the selection of N data objects with the greatest local outlier factor value is defined as contextual outliers. In the end, experimental results validate the effectiveness of the algorithm by using UCI data sets.

Initialization of K-modes clustering using outlier detection techniques

The K-modes clustering has received much attention, since it works well for categorical data sets. However, the performance of K-modes clustering is especially sensitive to the selection of initial cluster centers. Therefore, choosing the proper initial cluster centers is a key step for K-modes clustering. In this paper, we consider the initialization of K-modes clustering from the view of outlier detection. We present two different initialization algorithms for K-modes clustering, where the first is based on the traditional distance-based outlier detection technique, and the second is based on the partition entropy-based outlier detection technique. By using the above two outlier detection techniques to calculate the degree of outlierness of each object, our algorithms can guarantee that the chosen initial cluster centers are not outliers. Moreover, during the process of initialization, we adopt a new distance metric — weighted matching distance metric, to calculate the distance between two objects described by categorical attributes. Experimental results on several UCI data sets demonstrate the effectiveness of our initialization algorithms for K-modes clustering.

Spatial Outlier Detection of CO2 Monitoring Data Based on Spatial Local Outlier Factor

Spatial local outlier factor (SLOF) algorithm was adopted in this study for spatial outlier detection because of the limitations of the traditional static threshold detection. Based on the spatial characteristics of CO 2 monitoring data obtained in the carbon capture and storage (CCS) project, the K-Nearest Neighbour (KNN) graph was constructed using the latitude and longitude information of the monitoring points to identify the spatial neighbourhood of the monitoring points. Then SLOF was adopted to calculate the outlier degrees of the monitoring points and the 3σ rule was employed to identify the spatial outlier. Finally, the selection of K value was analysed and the optimal one was selected. The results show that, compared with the static threshold method, the proposed algorithm has a higher detection precision. It can overcome the shortcomings of the static threshold method and improve the accuracy and diversity of local outlier detection, which provides a reliable reference for the safety assessment and warning of CCS monitoring.

Mining multidimensional contextual outliers from categorical relational data

A wide range of methods have been proposed for detecting different types of outliers in both the full attribute space and its subspaces. However, the interpretability of outliers, that is, explaining in what ways and to what extent an object is an outlier, remains a critical issue. In this paper, we focus on improving the interpretability of outliers. Particularly, we develop a notion of multidimensional contextual outliers to model the context of an outlier, and propose a framework for contextual outlier detection. Intuitively, a contextual outlier is a small group of objects that share strong similarity with a significantly larger reference group of objects on some attributes, but deviate dramatically on some other attributes. In contextual outlier detection, we identify not only the outliers, but also their associated contextual information including (1) comparing to what reference group of objects the detected object(s) is/are an outlier; (2) the attributes defining the unusual behavior of the outlier(s) compared against the reference group; (3) the population of similar outliers sharing the same context; and (4) the outlier degree, which measures the population ratio between the reference group and the outlier group. We present an algorithm and conduct extensive experiments to evaluate our approach.

Absence d’avantage sur l’alignement frontal des prothèses de genou posées avec ancillaire personnalisé : revue systématique et méta-analyse

AimA systematic review and meta-analysis of clinical studies assessing alignment outcomes in patient- specific instrumented (PSI) knee arthroplasty was conducted. Materials and methodsPRISMA compliant data was extracted from literature databases up to January 2014. ResultsTwenty-six studies met the inclusion criteria, reporting a total of 1792 knees. Twenty-three studies reported alignment outcomes in the coronal plane, 11 in the sagittal plane. In all but three series, MRI was the pre-operative imaging modality. Range of mean post-operative alignment (Hip Knee Ankle [HKA] angle) was 176.5 to 181.70. The proportion of three degrees of outliers showed an overall mean of 18.6%. In total, fifteen studies compared alignment outcomes between standard and PSI. From these, four studies showed significantly higher accuracy of coronal plane alignment with PSI (HKAangle). Meta-analysis of seven high-quality comparative studies demonstrated no significant increased accuracy in post-operative mechanical axis (HKAangle) with PSI. Sub-group meta-analysis of both femoral and tibial rotation was not feasible due to a low number of inclusive high-quality series. ConclusionsFurther studies are required to demonstrate both clinical and radiological alignment outcomes in PSI knee arthroplasty with focus upon tibial and femoral rotation. Level of evidence1

Efficient Sparse ICP

The registration of two geometric surfaces is typically addressed using variants of the Iterative Closest Point (ICP) algorithm. The Sparse ICP method formulates the problem using sparsity-inducing norms, significantly improving the resilience of the registration process to large amounts of noise and outliers, but introduces a significant performance degradation. In this paper we first identify the reasons for this performance degradation and propose a hybrid optimization system that combines a Simulated Annealing search along with the standard Sparse ICP, in order to solve the underlying optimization problem more efficiently. We also provide several insights on how to further improve the overall efficiency by using a combination of approximate distance queries, parallel execution and uniform subsampling. The resulting method provides cumulative performance gain of more than one order of magnitude, as demonstrated through the registration of partially overlapping scans with various degrees of noise and outliers.

실내 비행 로봇을 위한 WPAN 기반 자가 측위 시스템 개발

As flying robots are becoming popular, there are increased needs to use themforsuch purposes as parcel delivery, serving in restaurants, and stage performances. To control flying robots such as quad copters, localization is essential. In order to properly position flying robots, many techniques are in development, including IR (infra-red)-based systemswhich catch markers on a flying robot in order that it can position itself. However, this technique demonstrates only short coverage. Furthermore, localization from inertial sensors diverges as time passes. For this reason, this paper suggests a TWR (two-way ranging) based positioning technique. Despite the weaknesses in currently available TWR system, this paper suggests a self-positioning and outlier detection technique in order to provide reliable position information with a faster update rate. The self-positioning system sends a shorter message which reduces wireless traffic. By detecting and removing outlier measurements, a positioning result with better accuracy is acquired. Finally, this paper shows that the suggesting system detects outlierssequentially from less than half the number of anchors in localization system according to the degree of outlier in measurement and the noise level. By performing an outlier algorithm, better positioning accuracy is acquired as shown in the experimental result.

A Novel Algorithm to Diagnosis Type II Diabetes Mellitus Based on Association Rule Mining Using MPSO-LSSVM with Outlier Detection Method

Background/Objectives: The execution of Frequent Pattern Growth algorithm on medical data is difficult. Association rule based classification is an interesting area focused that can be utilized for early diagnosis. Methods/Statistical analysis: Discretization phase is necessary to transform numerical characteristics. The results are given to Complete Frequent Patten Growth++ for the purpose of rule induction. Accordingly, using Modified Particle Swarm Optimization together with Least Squares Support Vector Machine scheme (MPSO-LSSVM) rules are produced with outlier detection method. Pima Indians Diabetes Data Set is taken as an input. The execution time, number of rules generation and the detection of outlier percentage are analyzed. Results: The CFP-growth algorithm utilizes for finding frequent patterns where constructing the Minimum Item Support (MIS)-tree, CFP-array and producing frequent patterns from the MIS-tree. From the set of frequent item sets found, create all the association rules that have a confidence exceeding the minimum confidence. The Enhanced outlier detection method is used for determining the outlier degree from association rules for outlier detection. Association rules are mined using MPSO-LSSVM classification based association rule mining algorithm. The classification based association rule generation using MPSO-LSSVM is utilized first time in this work with outlier detection method. For the reason of eradicating the effect of unavoidable outliers in investigation sample on a scheme’s performance, a new MPSO-LSSVM with the integration of outlier detection method is proposed first time. The experimental observations reveal that this framework provides a better accuracy of 95% when evaluated against the existing techniques. Conclusion/Application: CFP-Growth++ proposed for rule pruning and MPSO-LSSVM based algorithm used for mining association rules from Type-2 DM dataset. This work is suitable for early detection of type-2 diabetes mellitus disease.

No demonstrable benefit for coronal alignment outcomes in PSI knee arthroplasty: A systematic review and meta-analysis

AimA systematic review and meta-analysis of clinical studies assessing alignment outcomes in patient-specific instrumented (PSI) knee arthroplasty was conducted. Materials and methodsPRISMA compliant data was extracted from literature databases up to January 2014. ResultsTwenty-six studies met the inclusion criteria, reporting a total of 1792 knees. Twenty-three studies reported alignment outcomes in the coronal plane, 11 in the sagittal plane. In all but three series, MRI was the preoperative imaging modality. Range of mean postoperative alignment (hip–knee–ankle [HKA] angle) was 176.5 to 181.70. The proportion of three degrees of outliers showed an overall mean of 18.6%. In total, fifteen studies compared alignment outcomes between standard and PSI. From these, four studies showed significantly higher accuracy of coronal plane alignment with PSI (HKA angle). Meta-analysis of seven high-quality comparative studies demonstrated no significant increased accuracy in postoperative mechanical axis (HKA angle) with PSI. Subgroup meta-analysis of both femoral and tibial rotation was not feasible due to a low number of inclusive high-quality series. ConclusionsPSI knee arthroplasty is shown not to confer increased accuracy in reconstituting the postoperative mechanical axis. Further studies are required to demonstrate both clinical and radiological alignment outcomes in PSI knee arthroplasty with focus upon tibial and femoral rotation. Level of evidenceLevel 2 – meta-analysis.

Rough K-means Outlier Factor Based on Entropy Computation

Many studies of outlier detection have been developed based on the cluster-based outlier detection approach, since it does not need any prior knowledge of the dataset. However, the previous studies only regard the outlier factor computation with respect to a single point or a small cluster, which reflects its deviates from a common cluster. Furthermore, all objects within outlier cluster are assumed to be similar. The outlier objects intuitively can be grouped into the outlier clusters and the outlier factors of each object within the outlier cluster should be different gradually. It is not natural if the outlierness of each object within outlier cluster is similar. This study proposes the new outlier detection method based on the hybrid of the Rough K-Means clustering algorithm and the entropy computation. We introduce the outlier degree measure namely the entropy outlier factor for the cluster based outlier detection. The proposed algorithm sequentially finds the outlier cluster and calculates the outlier factor degree of the objects within outlier cluster. Each object within outlier cluster is evaluated using entropy cluster-based to a whole cluster. The performance of the algorithm has been tested on four UCI benchmark data sets and show outperform especially in detection rate.

Model-wise and point-wise random sample consensus for robust regression and outlier detection

Popular regression techniques often suffer at the presence of data outliers. Most previous efforts to solve this problem have focused on using an estimation algorithm that minimizes a robust M-estimator based error criterion instead of the usual non-robust mean squared error. However the robustness gained from M-estimators is still low. This paper addresses robust regression and outlier detection in a random sample consensus (RANSAC) framework. It studies the classical RANSAC framework and highlights its model-wise nature for processing the data. Furthermore, it introduces for the first time a point-wise strategy of RANSAC. New estimation algorithms are developed following both the model-wise and point-wise RANSAC concepts. The proposed algorithms’ theoretical robustness and breakdown points are investigated in a novel probabilistic setting. While the proposed concepts and algorithms are generic and general enough to adopt many regression machineries, the paper focuses on multilayered feed-forward neural networks in solving regression problems. The algorithms are evaluated on synthetic and real data, contaminated with high degrees of outliers, and compared to existing neural network training algorithms. Furthermore, to improve the time performance, parallel implementations of the two algorithms are developed and assessed to utilize the multiple CPU cores available on nowadays computers.

Trend analysis of categorical data streams with a concept change method

This paper proposes a new method to trend analysis of categorical data streams. A data stream is partitioned into a sequence of time windows and the records in each window are assumed to carry a number of concepts represented as clusters. A data labeling algorithm is proposed to identify the concepts or clusters of a window from the concepts of the preceding window. The expression of a concept is presented and the distance between two concepts in two consecutive windows is defined to analyze the change of concepts in consecutive windows. Finally, a trend analysis algorithm is proposed to compute the trend of concept change in a data stream over the sequence of consecutive time windows. The methods for measuring the significance of an attribute that causes the concept change and the outlier degrees of objects are presented to reveal the causes of concept change. Experiments on real data sets are presented to demonstrate the benefits of the trend analysis method.