Multivariate Outlier Detection Methods Research Articles

Truncated Robust Distance for Clinical Laboratory Safety Data Monitoring and Assessment

Laboratory safety data are routinely collected in clinical studies for safety monitoring and assessment. We have developed a truncated robust multivariate outlier detection method for identifying subjects with clinically relevant abnormal laboratory measurements. The proposed method can be applied to historical clinical data to establish a multivariate decision boundary that can then be used for future clinical trial laboratory safety data monitoring and assessment. Simulations demonstrate that the proposed method has the ability to detect relevant outliers while automatically excluding irrelevant outliers. Two examples from actual clinical studies are used to illustrate the use of this method for identifying clinically relevant outliers.

Interpretation of multivariate outliers for compositional data

Compositional data—and most data in geochemistry are of this type—carry relative rather than absolute information. For multivariate outlier detection methods this implies that not the given data but appropriately transformed data need to be used. We use the isometric logratio (ilr) transformation, which seems to be generally the most proper one for theoretical and practical reasons. In this space it is difficult to interpret the outliers, because the reason for outlyingness can be complex. Therefore we introduce tools that support the interpretation of outliers by representing multivariate information in biplots, maps, and univariate scatterplots.

Detection of multivariate outliers in business survey data with incomplete information

Many different methods for statistical data editing can be found in the literature but only few of them are based on robust estimates (for example such as BACON-EEM, epidemic algorithms (EA) and transformed rank correlation (TRC) methods of Beguin and Hulliger). However, we can show that outlier detection is only reasonable if robust methods are applied, because the classical estimates are themselves influenced by the outliers. Nevertheless, data editing is essential to check the multivariate data for possible data problems and it is not deterministic like the traditional micro editing where all records are extensively edited manually using certain rules/constraints. The presence of missing values is more a rule than an exception in business surveys and poses additional severe challenges to the outlier detection. First we review the available multivariate outlier detection methods which can cope with incomplete data. In a simulation study, where a subset of the Austrian Structural Business Statistics is simulated, we compare several approaches. Robust methods based on the Minimum Covariance Determinant (MCD) estimator, S-estimators and OGK-estimator as well as BACON-BEM provide the best results in finding the outliers and in providing a low false discovery rate. Many of the discussed methods are implemented in the R package $${\tt{rrcovNA}}$$ which is available from the Comprehensive R Archive Network (CRAN) at http://www.CRAN.R-project.org under the GNU General Public License.

Robust Multivariate Outlier Detection Methods for Environmental Data

Outliers are an inevitable concern that needs to be identified and dealt with whenever one analyzes a large data set. Today’s water quality data are often collected on different scales, encompass several sites, monitor several correlated parameters, involve a multitude of individuals from several agencies, and span over several years. As such, the ability to identify outliers, which may affect the results of the analysis, is crucial. This note presents several statistical techniques that have been developed to deal with this problem, with particular emphasis on robust multivariate methods. These techniques are capable of isolating outliers while overcoming the effects of masking that can hinder the effectiveness of common outlier detection techniques such as Mahalanobis distances (MD). This note uses a comprehensive national metadata set on lake water quality as a case study to analyze the effectiveness of three robust outlier detection techniques, namely, the minimum covariance determinant (MCD), the minimum volume ellipsoid (MVE), and M-estimators. The note compares the results generated from these three techniques to assess the severity of each method when it comes to labeling observations as outliers. The results demonstrate the limitations of using MD to analyze multidimensional water quality data. The analysis also highlighted the differences between the three robust multivariate methods, whereby the MVE method was found to be the most severe when it came to outlier detection, while the MCD was the most lenient. Of the three robust multivariate outlier detection methods analyzed, the M-estimator proved to be the most flexible because it allowed for downweighting rather than censoring many borderline outlier observations.

Improvement of identification of blast furnace ironmaking process by outlier detection and missing value imputation

The control of blast furnace ironmaking process requires model of process dynamics accurate enough to facilitate the control strategies. However, data sets collected from blast furnace contain considerable number of missing values and outliers. These values can significantly affect subsequent statistical analysis and thus the identification of the whole process, so it becomes much important to deal with these values. This paper considers a data processing procedure including missing value imputation and outlier detection, and examines the impact of processing to the identification of blast furnace ironmaking process. Missing values are imputed based on the decision tree algorithm and outliers are identified and discarded through a set of multivariate outlier detection methods. The data sets before and after processing are then used for identification. Two classic identification methods, N4SID (numerical algorithms for state space subspace system identification) and PEM (prediction error method) are considered and a comparative study is presented.

Separating a mixture of two normals with proportional covariances

We propose a simple affine equivariant clustering method, based on the idea of best linear classification, for samples from a mixture of two multivariate normal distributions with different mean vectors but proportional covariance matrices. To ameliorate the curse of dimensionality, a non-parametric approach to find candidates for a best linear discriminant function is presented. By using simulation studies and a real example, we show that for large samples in high dimensions, the proposed method can be a useful supplement to general-purpose multivariate outlier detection methods.

Robust principal components: A generalized variance perspective

This article compares several methods for performing robust principal component analysis, two of which have not been considered in previous articles. The criterion here, unlike that of extant articles aimed at comparing methods, is how well a method maximizes a robust version of the generalized variance of the projected data. This is in contrast to maximizing some measure of scatter associated with the marginal distributions of the projected scores, which does not take into account the overall structure of the projected data. Included are comparisons in which distributions are not elliptically symmetric. One of the new methods simply removes outliers using a projection-type multivariate outlier detection method that has been found to perform well relative to other outlier detection methods that have been proposed. The other new method belongs to the class of projection pursuit techniques and differs from other projection pursuit methods in terms of the function it tries to maximize. The comparisons include the method derived by Maronna (2005), the spherical method derived by Locantore et al. (1999), as well as a method proposed by Hubert, Rousseeuw, and Vanden Branden (2005). From the perspective used, the method by Hubert et al. (2005), the spherical method, and one of the new methods dominate the method derived by Maronna.

A Comparison of Multivariate Outlier Detection Methods for Clinical Laboratory Safety Data

During a clinical trial of a new treatment, a large number of variables are measured to monitor the safety of the treatment. It is important to detect outlying observations which may indicate that something abnormal is happening. To do this effectively, techniques are needed for finding multivariate outliers. Six techniques of this sort are described and illustrated on a typical laboratory safety data set. Their properties are investigated more thoroughly by means of a simulation study. The results show that some methods do better than others depending on whether or not the data set is multivariate normal, the dimension of the data set, the type of outlier, the proportion of outliers in a data set and the degree of contamination, i.e. ‘outlyingness’. The results indicate that it is desirable to run a battery of multivariate methods on a particular data set in an attempt to highlight possible outliers.

Multivariate outlier detection applied to multiply imputed laboratory data

In clinical laboratory safety data, multivariate outlier detection methods may highlight a patient whose laboratory measurements do not follow the same pattern of relationships as the majority of patients, although their individual measurements are not found to be outlying when considered one at a time. Missing data problems are often dealt with by imputing a single value as an estimate of the missing value. The completed data set may then be analysed using traditional methods. A disadvantage of using single imputation is the underestimation of variability, with a corresponding distortion of power in hypothesis testing. Multiple imputation methods attempt to overcome this problem, and in this paper a study is described which considers the application of multivariate outlier detection methods to multiply imputed clinical laboratory safety data sets. Three different proportions of missing data are generated in laboratory data sets of dimensions 4, 7, 12 and 30, and a comparison of eight multiple imputation methods is carried out. Two outlier detection techniques, Mahalanobis distance and generalized principal component analysis, are applied to the multiply imputed data sets, and their performances are discussed. Measures are introduced for assessing the accuracy of the missing data results, depending on which method of analysis is used. Copyright © 1999 John Wiley & Sons, Ltd.