Outlier Score Research Articles

The Outlier Interval Detection Algorithms on Astronautical Time Series Data

The Outlier Interval Detection is a crucial technique to analyze spacecraft fault, locate exception, and implement intelligent fault diagnosis system. The paper proposes two OID algorithms on astronautical Time Series Data, that is, variance based OID (VOID) and FFT andknearest Neighbour based OID (FKOID). The VOID algorithm divides TSD into many intervals and measures each interval’s outlier score according to its variance. This algorithm can detect the outlier intervals with great fluctuation in the time domain. It is a simple and fast algorithm with less time complexity, but it ignores the frequency information. The FKOID algorithm extracts the frequency information of each interval by means of Fast Fourier Transform, so as to calculate the distances between frequency features, and adopts the KNN method to measure the outlier score according to the sum of distances between the interval’s frequency vector and theKnearest frequency vectors. It detects the outlier intervals in a refined way at an appropriate expense of the time and is valid to detect the outlier intervals in both frequency and time domains.

Characteristics and classification of outlier detection techniques for wireless sensor networks in harsh environments: a survey

Wireless sensor networks (WSNs) have received considerable attention for multiple types of applications. In particular, outlier detection in WSNs has been an area of vast interest. Outlier detection becomes even more important for the applications involving harsh environments, however, it has not received extensive treatment in the literature. The identification of outliers in WSNs can be used for filtration of false data, find faulty nodes and discover events of interest. This paper presents a survey of the essential characteristics for the analysis of outlier detection techniques in harsh environments. These characteristics include, input data type, spatio-temporal and attribute correlations, user specified thresholds, outlier types(local and global), type of approach(distributed/centralized), outlier identification(event or error), outlier degree, outlier score, susceptibility to dynamic topology, non-stationarity and inhomogeneity. Moreover, the prioritization of various characteristics has been discussed for outlier detection techniques in harsh environments. The paper also gives a brief overview of the classification strategies for outlier detection techniques in WSNs and discusses the feasibility of various types of techniques for WSNs deployed in harsh environments.

Cluster based Outlier Detection

detection is a fundamental issue in data mining, specifically it has been used to detect and remove anomalous objects from data.mining. The proposed approach to detect outlier includes three methods which are clustering, pruning and computing outlier score. For clustering k-means algorithm is used which partition the dataset into given number of clusters. In pruning, based on some distance measure, points which are closed to centroid of each cluster are pruned. For the unpruned points, local distance based outlier factor (LDOF) measure is calculated. A measure called LDOF, tells how much a point is deviating from its neighbors. The high LDOF value of a point indicates that the point is deviating more from its neighbors and probably it may be an outlier. General Terms Outlier, cluster, pruning, outlier score, k nearest neighbor KeywordsLDOF

Detection of Outlier Residues for Improving Interface Prediction in Protein Heterocomplexes

Sequence-based understanding and identification of protein binding interfaces is a challenging research topic due to the complexity in protein systems and the imbalanced distribution between interface and noninterface residues. This paper presents an outlier detection idea to address the redundancy problem in protein interaction data. The cleaned training data are then used for improving the prediction performance. We use three novel measures to describe the extent a residue is considered as an outlier in comparison to the other residues: the distance of a residue instance from the center instance of all residue instances of the same class label (Dist), the probability of the class label of the residue instance (PCL), and the importance of within-class and between-class (IWB) residue instances. Outlier scores are computed by integrating the three factors; instances with a sufficiently large score are treated as outliers and removed. The data sets without outliers are taken as input for a support vector machine (SVM) ensemble. The proposed SVM ensemble trained on input data without outliers performs better than that with outliers. Our method is also more accurate than many literature methods on benchmark data sets. From our empirical studies, we found that some outlier interface residues are truly near to noninterface regions, and some outlier noninterface residues are close to interface regions.

The Story of an Outlier: … A Case Study of One Young Deaf Child and His Journey Towards Early Mathematical Competence

The purpose of this study was to examine the story behind one young deaf child who demonstrated exceptional early mathematical competence and the individual and family characteristics that might have contributed to his success. As an outlier, this child achieved a score on a test of early mathematical ability that was two standard deviations higher than the next highest score. Using qualitative methodology, this study examines factors that may have contributed to this child's early mathematical success including family, home, and individual learning behaviours such as engagement, resilience, curiosity, and challenge seeking. Findings indicate that there is value to examining the story behind outlier scores and that multiple factors are likely to contribute to success.

An experimental comparison of real and artificial deception using a deception generation model

To develop a data mining approach for a deception application, data collection costs can be prohibitive because both deceptive data and truthful data are necessary to be collected. To reduce data collection costs, artificially generated deception data can be used, but the impact of using artificially generated deception data is not well understood. To study the relationship between artificial and real deception, this paper presents an experimental comparison using a novel deception generation model. The deception and truth data were collected from financial aid applications, a document centric area with limited resources for verification. The data collection provided a unique data set containing truth, natural deception, and boosted deception. To simulate deception, the Application Deception Model was developed to generate artificial deception in different deception scenarios. To study differences between artificial and real deception, an experiment was performed using deception level and data generation method as factors and directed distance and outlier score as outcome variables. Our results provided evidence of a reasonable similarity between artificial and real deception, suggesting the possibility of using artificially generated deception to reduce the costs associated with obtaining training data.

Increased promoter methylation in exfoliated breast epithelial cells in women with a previous breast biopsy

Accurately identifying women at increased risk of developing breast cancer will provide greater opportunity for early detection and prevention. DNA promoter methylation is a promising biomarker for assessing breast cancer risk. Breast milk contains large numbers of exfoliated epithelial cells that are ideal for methylation analyses. Exfoliated epithelial cells were isolated from the milk obtained from each breast of 134 women with a history of a non-proliferative benign breast biopsy (Biopsy Group). Promoter methylation of three tumor suppressor genes, RASSF1, SFRP1 and GSTP1, was assessed by pyrosequencing of bisulfite-modified DNA. Methylation scores from the milk of the 134 women in the Biopsy Group were compared to scores from 102 women for whom a breast biopsy was not a recruitment requirement (Reference Group). Mean methylation scores for RASSF1 and GSTP1 were significantly higher in the Biopsy than in the Reference Group. For all three genes the percentage of outlier scores was greater in the Biopsy than in the Reference Group but reached statistical significance only for GSTP1. A comparison between the biopsied and non-biopsied breasts of the Biopsy Group revealed higher mean methylation and a greater number of outlier scores in the biopsied breast for both SFRP1 and RASSF1, but not for GSTP1. This is the first evidence of CpG island methylation in tumor suppressor genes of women who may be at increased risk of developing breast cancer based on having had a prior breast biopsy.

Substantiating Anomalies In Wireless Networks Using Group Outlier Scores

Huge amounts of network traces can be collected from today's busy computer networks. Analyzing these traces could pave the way to detect unusual conditions and/or other anomalies. Presently, due to the lack of effective substantiating mechanisms intrusion detection systems often exhibit numerous false positives or negatives. The efficiency of a network intrusion detection system (NIDS) depends very much on detecting and effectively validating the de- tected anomalies. Furthermore, most NIDSs do not have proven mechanisms that will easily accommodate legitimate dynamic changes. Achieving dynamic adaptation in real time has been a long standing desire for effective intrusion detection and prevention. Real time detection of outliers is a feasible option to substantiate anomalies in large data sets, leading to effective intrusion detection and prevention. In this context we propose and investigate a novel mechanism to detect intruders and to classify security threats using group outliers. Our system monitors for timing and/or behavioral anomalies and uses outlier based techniques to substantiate the anomaly. In this paper we introduce the concept of Group Outlier Score (GOS) and its use in substantiating security threats in wireless networks. We have tested the concept on our experimental wireless networking environment. The analysis of the results reveals that with a threshold value of 1.2 for GOS our system demonstrates optimum performance.

Statistical outlier detection using direct density ratio estimation

We propose a new statistical approach to the problem of inlier-based outlier detection, i.e., finding outliers in the test set based on the training set consisting only of inliers. Our key idea is to use the ratio of training and test data densities as an outlier score. This approach is expected to have better performance even in high-dimensional problems since methods for directly estimating the density ratio without going through density estimation are available. Among various density ratio estimation methods, we employ the method called unconstrained least-squares importance fitting (uLSIF) since it is equipped with natural cross-validation procedures, allowing us to objectively optimize the value of tuning parameters such as the regularization parameter and the kernel width. Furthermore, uLSIF offers a closed-form solution as well as a closed-form formula for the leave-one-out error, so it is computationally very efficient and is scalable to massive datasets. Simulations with benchmark and real-world datasets illustrate the usefulness of the proposed approach.

Patient feedback for medical students

Patient feedback for medical students

Evaluation of alternative procedures for the provision of consensus and assigned values to sixteen trace elements in natural water used for an ICP-MS proficiency testing exercise

The University of Oviedo, in collaboration with the company ISC-Science, has conducted a proficiency testing exercise for Spanish ICP-MS users focused on trace elements determination in drinking waters. A total of 47 laboratories participated in this exercise where 16 elements (Al, Sb, As, B, Cd, Cu, Cr, Fe, Mn, Ni, Pb, Se, Sr, Mo, U, and Zn) were determined in a natural water sample. The analytical procedure used at the University of Oviedo was validated by analysis of the certified reference material SLRS-4 (riverine water) from the National Research Council of Canada. Assigned values were provided for 12 multi-isotopic elements by isotope dilution analysis. For this purpose, isotopically enriched spikes (10B, 53Cr, 57Fe, 61Ni, 65Cu, 67Zn, 77Se, 86Sr, 95Mo, 111Cd, 123Sb and 207Pb) were characterised by reverse isotope dilution analysis both by multicollector ICP-MS and collision cell ICP-MS (10B and 77Se). Finally, the collision cell instrument was adopted for the provision of the assigned values taking into account the results from the homogeneity and stability studies on the natural water sample. Considering the differences in expertise between the different laboratories, consensus values for all sixteen measured elements were calculated using alternative statistical tests for outlier removal. It was observed that the combination of the Cochran and Hampel tests provided the best agreement between the consensus and the assigned values. Finally, the analytical results from the different laboratories were evaluated by different scoring procedures taking into account the requirements of European Directive 98/83/EC for drinking water testing. As these types of exercises will be used by many laboratories to seek or maintain accreditation, in intercomparison exercises where consensus values are usually applied, the evaluated procedures for outlier elimination and scoring are critically compared.

Detecting outlier samples in multivariate time series dataset

Multivariate time series (MTS) samples which differ significantly from other MTS samples are referred to as outlier samples. In this paper, an algorithm designed to efficiently detect the top n outlier samples in MTS dataset, based on Solving Set, is proposed. An extended Frobenius Norm is used to compute the distance between MTS samples. The outlier score of MTS sample is the sum of the distances from its k nearest neighbors. The time complexity of the algorithm is subquadratic. We conduct experiments on two real-world datasets, stock market dataset and BCI (Brain Computer Interface) dataset. The experiment results show the efficiency and effectiveness of the algorithm.

OutRank: A GRAPH-BASED OUTLIER DETECTION FRAMEWORK USING RANDOM WALK

This paper introduces a stochastic graph-based algorithm, called OutRank, for detecting outliers in data. We consider two approaches for constructing a graph representation of the data, based on the object similarity and number of shared neighbors between objects. The heart of this approach is the Markov chain model that is built upon this graph, which assigns an outlier score to each object. Using this framework, we show that our algorithm is more robust than the existing outlier detection schemes and can effectively address the inherent problems of such schemes. Empirical studies conducted on both real and synthetic data sets show that significant improvements in detection rate and false alarm rate are achieved using the proposed framework.

Outlier Detection in Test and Questionnaire Data

Classical methods for detecting outliers deal with continuous variables. These methods are not readily applicable to categorical data, such as incorrect/correct scores (0/1) and ordered rating scale scores (e.g., 0, …, 4) typical of multi-item tests and questionnaires. This study proposes two definitions of outlier scores suited for categorical data. One definition combines information on outliers from scores on all the items in the test, and the other definition combines information from all pairs of item scores. For a particular item-score vector, an outlier score expresses the degree to which the item-score vector is unusual. For ten real-data sets, the distribution of each of the two outlier scores is inspected by means of Tukey's fences and the extreme studentized deviate procedure. It is investigated whether the outliers that are identified are influential with respect to the statistical analysis performed on these data. Recommendations are given for outlier identification and accommodation in test and questionnaire data.

Factors Associated With Stigma Among Caregivers of Patients With Bipolar Disorder in the STEP-BD Study

Factors Associated With Stigma Among Caregivers of Patients With Bipolar Disorder in the STEP-BD Study

Factors Associated With Stigma Among Caregivers of Patients With Bipolar Disorder in the STEP-BD Study

Little is known about the factors contributing to mental illness stigma among caregivers of people with bipolar disorder. A total of 500 caregivers of patients participating in the Systematic Treatment Enhancement Program for Bipolar Disorder (STEP-BD) study were interviewed in a cross-sectional design on measures of stigma, mood, burden, and coping. Relatives and friends with bipolar disorder were assessed on measures of diagnosis and clinical status, determined by a days-well measure derived from psychiatrist ratings of DSM-IV episode status. Because patients' clinical status varied widely, separate models were run for patients who were euthymic for at least three-fourths of the past year (well group) and for those who met criteria for an affective episode for at least one-fourth of the previous year (unwell group). Stepwise multiple regression was used to identify patient, illness, and caregiver characteristics associated with caregiver stigma. In the unwell group, greater mental illness stigma was associated with bipolar I (versus II) disorder, less social support for the caregiver, fewer caregiver social interactions, and being a caregiver of Hispanic descent. In the well group, greater stigma was associated with being a caregiver who is the adult child of a parent with bipolar disorder, who has a college education, who has fewer social interactions, and who cares for a female bipolar patient. Mental illness stigma was found to be prevalent among caregivers of persons with bipolar disorder who have active symptoms as well as for caregivers of those who have remitted symptoms. Stigma is typically associated with factors identifying patients as "different" during symptomatic periods. Research is needed to understand how the stigma experienced by caregivers during stable phases of illness differs from the stigma experienced during patients' illness states.

FINDING OUTLIERS AT MULTIPLE SCALES

Outlier detection targets those exceptional data whose pattern is rare and lie in low density regions. In this paper, under the assumption of complete spatial randomness inside clusters, we propose an MDV (Multi-scale Deviation of the Volume) approach to identifying outliers. In addition to assigning an outlier score for each object, it directly outputs a crisp outlier set. It also offers a plot showing the data structure in every object's vicinity, which is useful in explaining why it may be outlying. Finally, the effectiveness of MDV is demonstrated with both artificial and real datasets.

A test of enhancing model accuracy in high-throughput crystallography

The high throughput of structure determination pipelines relies on increased automation and, consequently, a reduction of time spent on interactive quality control. In order to meet and exceed current standards in model accuracy, new approaches are needed for the facile identification and correction of model errors during refinement. One such approach is provided by the validation and structure-improvement tools of the MOL: PROBITY: web service. To test their effectiveness in high-throughput mode, a large subset of the crystal structures from the SouthEast Collaboratory for Structural Genomics (SECSG) has used protocols based on the MOL: PROBITY: tools. Comparison of 29 working-set and 19 control-set SECSG structures shows that working-set outlier scores for updated Ramachandran-plot, sidechain rotamer, and all-atom steric criteria have been improved by factors of 5- to 10-fold (relative to the control set or to a Protein Data Bank sample), while quality of covalent geometry, R(work), R(free), electron density and difference density are maintained or improved. Some parts of this correction process are already fully automated; other parts involve manual rebuilding of conformations flagged by the tests as trapped in the wrong local minimum, often altering features of functional significance. The ease and effectiveness of this technique shows that macromolecular crystal structures from either traditional or high-throughput determinations can feasibly reach a new level of excellence in conformational accuracy and reliability.

An outlier-based data association method for linking criminal incidents

Serial criminals are a major threat in the modern society. Associating incidents committed by the same offender is of great importance in studying serial criminals. In this paper, we present a new outlier-based approach to resolve this criminal incident association problem. In this approach, criminal incident data are first modeled into a number of cells, and then a measurement function, called outlier score function, is defined over these cells. Incidents in a cell are determined to be associated with each other when the score is significant enough. We applied our approach to a robbery dataset from Richmond, VA. Results show that this method can effectively solve the criminal incident association problem.

Outlier Detection And Data Association For Data Mining Criminal Incidents

Outlier detection has been extensively studied in the field of statistics, and a number of discordancy tests have been developed. Most of these studies treat outliers as “noise” and they try to eliminate the effects of outliers by removing outliers or develop some outlier-resistant methods. However, in data mining, we consider outliers “meaningful input signals” rather than “noise”. In some cases, outliers represent unique characteristics of the objects, which are important to an organization. Law enforcement is one area where outlier detection is critically important. In law enforcement, we want to associate criminal incidents caused by the same persoflgroup and detect outliers from this behavior. The purpose of this paper is two-folded: to describe an outlier detection technique and to propose a data association method based upon this technique. We focus our analysis on categorical data since these data are typically found in crime analysis. First, we develop an outlier score function, which measures the “extremeness” level for observations. Then we dkcuss some of the properties of this outlier score function. Finally, we describe a data association method based on the outlier score function. We apply our data association method to a real robbery data from the Richmond, Virginia, USA. Results show that the approach is promising.