Graphical Diagnostic Tools Research Articles

Robust discriminant analysis

AbstractDiscriminant analysis (DA) is one of the most popular methods for classification due to its conceptual simplicity, low computational cost, and often solid performance. In its standard form, DA uses the arithmetic mean and sample covariance matrix to estimate the center and scatter of each class. We discuss and illustrate how this makes standard DA very sensitive to suspicious data points, such as outliers and mislabeled cases. We then present an overview of techniques for robust DA, which are more reliable in the presence of deviating cases. In particular, we review DA based on robust estimates of location and scatter, along with graphical diagnostic tools for visualizing the results of DA.This article is categorized under: Statistical and Graphical Methods of Data Analysis > Robust Methods Statistical Learning and Exploratory Methods of the Data Sciences > Clustering and Classification

GENIUS-MAWII: for robust Mendelian randomization with many weak invalid instruments.

Mendelian randomization (MR) addresses causal questions using genetic variants as instrumental variables. We propose a new MR method, G-Estimation under No Interaction with Unmeasured Selection (GENIUS)-MAny Weak Invalid IV, which simultaneously addresses the 2 salient challenges in MR: many weak instruments and widespread horizontal pleiotropy. Similar to MR-GENIUS, we use heteroscedasticity of the exposure to identify the treatment effect. We derive influence functions of the treatment effect, and then we construct a continuous updating estimator and establish its asymptotic properties under a many weak invalid instruments asymptotic regime by developing novel semiparametric theory. We also provide a measure of weak identification, an overidentification test, and a graphical diagnostic tool.

A comprehensive decision support system for the characterization of probability distribution tails for daily precipitation

A conventional Decision Support System (DSS) can be used to characterize tails of probability distributions into distribution families using various graphical methods. Existing DSSs lack efficient segregation of the Lognormal distribution from the Regularly varying and Subexponential distribution families. Also, they lack the ability to identify the distributions from the hyper-exponential distributions. Recently developed graphical diagnostic tools, such as concentration profile, concentration adjusted expected shortfall, discriminant moment ratio plot, maximum-to-sum plot, and Zenga plot can classify the tails of distributions into various classes if used in an appropriate order in combination with tools of conventional DSS. The present study proposes a comprehensive DSS that alleviates the shortcomings associated with the conventional DSS and characterizes the tails of distributions into classes B\\A (Pareto type), C\\B (regularly varying), D\\C (subexponential), E (Exponential type), hyper-exponential class (outside class E) and LN (Lognormal) distribution (the limiting case between class C and D). The robustness of the proposed DSS over the conventional DSS is established through a simulation experiment. Further, this study also evaluates the influence of the sample size on the effective implementation of the proposed DSS. Finally, the proposed DSS is applied to characterize the tails of daily gridded precipitation data over India. Results indicate that precipitation data from about 98% of grids over India exhibit distributions from heavy-tailed families. The study recommends the use of heavy-tailed distributions to model daily precipitation data over India. The study also suggests that one should rely on more than one graphical method for deducing rational conclusions regarding tail characterization.

Graphical and numerical diagnostic tools to assess multiple imputation models by posterior predictive checking

ProblemThe congenial of the imputation model is crucial for valid statistical inferences. Hence, it is important to develop methodologies for diagnosing imputation models. AimWe propose and evaluate a new diagnostic method based on posterior predictive checking to diagnose the congeniality of fully conditional imputation models. Our method applies to multiple imputation by chained equations, which is widely used in statistical software. MethodsThe proposed method compares the observed data with their replicates generated under the corresponding posterior predictive distributions to diagnose the performance of imputation models. The method applies to various imputation models, including parametric and semi-parametric approaches and continuous and discrete incomplete variables. We studied the validity of the method through simulation and application. ResultsThe proposed diagnostic method based on posterior predictive checking demonstrates its validity in assessing the performance of imputation models. The method can diagnose the consistency of imputation models with the substantive model and can be applied to a broad range of research contexts. ConclusionThe diagnostic method based on posterior predictive checking provides a valuable tool for researchers who use fully conditional specification to handle missing data. By assessing the performance of imputation models, our method can help researchers improve the accuracy and reliability of their analyzes. Furthermore, our method applies to different imputation models. Hence, it is a versatile and valuable tool for researchers identifying plausible imputation models.

Transparency of high-dimensional propensity score analyses: Guidance for diagnostics and reporting.

PurposeThe high‐dimensional propensity score (HDPS) is a semi‐automated procedure for confounder identification, prioritisation and adjustment in large healthcare databases that requires investigators to specify data dimensions, prioritisation strategy and tuning parameters. In practice, reporting of these decisions is inconsistent and this can undermine the transparency, and reproducibility of results obtained. We illustrate reporting tools, graphical displays and sensitivity analyses to increase transparency and facilitate evaluation of the robustness of analyses involving HDPS.MethodsUsing a study from the UK Clinical Practice Research Datalink that implemented HDPS we demonstrate the application of the proposed recommendations.ResultsWe identify seven considerations surrounding the implementation of HDPS, such as the identification of data dimensions, method for code prioritisation and number of variables selected. Graphical diagnostic tools include assessing the balance of key confounders before and after adjusting for empirically selected HDPS covariates and the identification of potentially influential covariates. Sensitivity analyses include varying the number of covariates selected and assessing the impact of covariates behaving empirically as instrumental variables. In our example, results were robust to both the number of covariates selected and the inclusion of potentially influential covariates. Furthermore, our HDPS models achieved good balance in key confounders.ConclusionsThe data‐adaptive approach of HDPS and the resulting benefits have led to its popularity as a method for confounder adjustment in pharmacoepidemiological studies. Reporting of HDPS analyses in practice may be improved by the considerations and tools proposed here to increase the transparency and reproducibility of study results.

Multiple imputation regression discontinuity designs: Alternative to regression discontinuity designs to estimate the local average treatment effect at the cutoff

The regression discontinuity design (RDD) is one of the most credible methods for causal inference that is often regarded as a missing data problem in the potential outcomes framework. However, the methods for missing data such as multiple imputation are rarely used as a method for causal inference. This article proposes multiple imputation regression discontinuity designs (MIRDDs), an alternative way of estimating the local average treatment effect at the cutoff point by multiply-imputing potential outcomes. To assess the performance of the proposed method, Monte Carlo simulations are conducted under 112 different settings, each repeated 5,000 times. The simulation results show that MIRDDs perform well in terms of bias, root mean squared error, coverage, and interval length compared to the standard RDD method. Also, additional simulations exhibit promising results compared to the state-of-the-art RDD methods. Finally, this article proposes to use MIRDDs as a graphical diagnostic tool for RDDs. We illustrate the proposed method with data on the incumbency advantage in U.S. House elections. To implement the proposed method, an easy-to-use software program is also provided.

On multivariate copula modeling of dependent degradation processes

Multivariate degradation processes have been observed in many engineering systems. Most existing multivariate degradation modeling techniques, such as multivariate general path models or multivariate Wiener process models, assume an underlying Gaussian dependence structure. Unfortunately, in reality, the dependencies among degradation processes are often nonlinear, asymmetric and greatly tail-skewed, and thus limit the usefulness of the conventional modeling techniques in practice. To overcome these limitations, in this paper, we develop a copula-based multivariate modeling framework. Three fundamental copula classes are applied to model the complex dependence structure among correlated degradation processes. Statistical inference and model selection techniques, including two graphical diagnostic tools, a test of independence and a goodness-of-fit test, are employed to identify the best model. The advantages of the proposed modeling framework are demonstrated through simulation studies. And we also discuss the effect of ignoring tail dependence on system failure probability assessment. Finally, the applications of the copula-based multivariate degradation models on both system reliability evaluation and remaining useful life prediction are provided. The proposed methodology is illustrated using a numerical example.

Parametric mode regression for bounded responses.

We propose new parametric frameworks of regression analysis with the conditional mode of a bounded response as the focal point of interest. Covariate effects estimation and prediction based on the maximum likelihood method under two new classes of regression models are demonstrated. We also develop graphical and numerical diagnostic tools to detect various sources of model misspecification. Predictions based on different central tendency measures inferred using various regression models are compared using synthetic data in simulations. Finally, we conduct regression analysis for data from the Alzheimer's Disease Neuroimaging Initiative to demonstrate practical implementation of the proposed methods. Supporting Information that contain technical details and additional simulation and data analysis results are availableonline.

Modelling average maximum daily temperature using r largest order statistics: An application to South African data.

Natural hazards (events that may cause actual disasters) are established in the literature as major causes of various massive and destructive problems worldwide. The occurrences of earthquakes, floods and heat waves affect millions of people through several impacts. These include cases of hospitalisation, loss of lives and economic challenges. The focus of this study was on the risk reduction of the disasters that occur because of extremely high temperatures and heat waves. Modelling average maximum daily temperature (AMDT) guards against the disaster risk and may also help countries towards preparing for extreme heat. This study discusses the use of the r largest order statistics approach of extreme value theory towards modelling AMDT over the period of 11 years, that is, 2000–2010. A generalised extreme value distribution for r largest order statistics is fitted to the annual maxima. This is performed in an effort to study the behaviour of the r largest order statistics. The method of maximum likelihood is used in estimating the target parameters and the frequency of occurrences of the hottest days is assessed. The study presents a case study of South Africa in which the data for the non-winter season (September–April of each year) are used. The meteorological data used are the AMDT that are collected by the South African Weather Service and provided by Eskom. The estimation of the shape parameter reveals evidence of a Weibull class as an appropriate distribution for modelling AMDT in South Africa. The extreme quantiles for specified return periods are estimated using the quantile function and the best model is chosen through the use of the deviance statistic with the support of the graphical diagnostic tools. The Entropy Difference Test (EDT) is used as a specification test for diagnosing the fit of the models to the data.

A shared parameter model of longitudinal measurements and survival time with heterogeneous random-effects distribution

ABSTRACTTypical joint modeling of longitudinal measurements and time to event data assumes that two models share a common set of random effects with a normal distribution assumption. But, sometimes the underlying population that the sample is extracted from is a heterogeneous population and detecting homogeneous subsamples of it is an important scientific question. In this paper, a finite mixture of normal distributions for the shared random effects is proposed for considering the heterogeneity in the population. For detecting whether the unobserved heterogeneity exits or not, we use a simple graphical exploratory diagnostic tool proposed by Verbeke and Molenberghs [34] to assess whether the traditional normality assumption for the random effects in the mixed model is adequate. In the joint modeling setting, in the case of evidence against normality (homogeneity), a finite mixture of normals is used for the shared random-effects distribution. A Bayesian MCMC procedure is developed for parameter estimation and inference. The methodology is illustrated using some simulation studies. Also, the proposed approach is used for analyzing a real HIV data set, using the heterogeneous joint model for this data set, the individuals are classified into two groups: a group with high risk and a group with moderate risk.

Graphical and numerical diagnostic tools to assess suitability of multiple imputations and imputation models.

Multiple imputation has become a popular approach for analyzing incomplete data. Many software packages are available to multiply impute the missing values and to analyze the resulting completed data sets. However, diagnostic tools to check the validity of the imputations are limited, and the majority of the currently available methods need considerable knowledge of the imputation model. In many practical settings, however, the imputer and the analyst may be different individuals or from different organizations, and the analyst model may or may not be congenial to the model used by the imputer. This article develops and evaluates a set of graphical and numerical diagnostic tools for two practical purposes: (i) for an analyst to determine whether the imputations are reasonable under his/her model assumptions without actually knowing the imputation model assumptions; and (ii) for an imputer to fine tune the imputation model by checking the key characteristics of the observed and imputed values. The tools are based on the numerical and graphical comparisons of the distributions of the observed and imputed values conditional on the propensity of response. The methodology is illustrated using simulated data sets created under a variety of scenarios. The examples focus on continuous and binary variables, but the principles can be used to extend methods for other types of variables. Copyright © 2016 John Wiley & Sons, Ltd.

Discussion of “Multivariate functional outlier detection”

This interesting paper adds on the authors long and outstanding trajectory in robust statistics and , more specifically, on robust functional data analysis. We congratulate Mia Hubert, Peter Rousseeuw and Pieter Segaert for this important contribution. As the authors point out, most of the literature to date on functional outlier detection deals with univariate functional data (one curve observed by individual). This work considers the case of p-variate functional data (p curves observed by individual). The paper discusses carefully the problem of outlier detection in this setting and starts by establishing a classification of different outlying behaviours. Then, several p-variate functional depths and distance functions are defined by integrating over time the existing or newly defined p-variate counterparts. Finally, by combining these measures, several graphical diagnostic tools are proposed. We would like to contribute to the discussion by focusing on two aspects. Firstly, we will compare the proposed taxonomy of functional outliers with the classification currently adopted in the literature. Secondly, we will comment on the differences between the proposed collection of methods and the outliergram (Arribas-Gil and Romo 2014), the recent procedure to detect shape outliers. We compare it with the proposed methodology in several examples.

Diagnostic tools for bivariate accelerated life regression models.

We propose graphical diagnostic tools to assess the fit of a bivariate accelerated lifetime regression model. Using univariate residuals for each response measurement in a pair, we assess their dependence structure via the bivariate probability integral transformation of univariate residuals, which we call V-residuals. To reduce the computational burden associated with plots of V-residuals, as well as some uncertainty associated with parameter estimation, we next develop K-residuals. We also devise adjusted V- and K-residuals to account for right censoring of any response. Via simulation studies, we examine the statistical behaviour of Q-Q plots of the estimated K-residuals, and demonstrate the potential of these plots to identify an appropriate choice of frailty distribution in bivariate modeling. We apply our proposed diagnostic tools to the Diabetic Retinopathy Study and assess the goodness of fit of various models fitted to these study data using different choices of frailty distributions combined with several baseline survivor functions.

Detecting and visualizing outliers in provider profiling via funnel plots and mixed effect models

In this work we propose the use of a graphical diagnostic tool (the funnel plot) to detect outliers among hospitals that treat patients affected by Acute Myocardial Infarction (AMI). We consider an application to data on AMI hospitalizations recorded in the administrative databases of our regional district. The outcome of interest is the in-hospital mortality, a variable indicating if the patient has been discharged dead or alive. We then compare the results obtained by graphical diagnostic tools with those arising from fitting parametric mixed effects models to the same data.

The gradient function as an exploratory goodness-of-fit assessment of the random-effects distribution in mixed models

Inference in mixed models is often based on the marginal distribution obtained from integrating out random effects over a pre-specified, often parametric, distribution. In this paper, we present the so-called gradient function as a simple graphical exploratory diagnostic tool to assess whether the assumed random-effects distribution produces an adequate fit to the data, in terms of marginal likelihood. The method does not require any calculations in addition to the computations needed to fit the model, and can be applied to a wide range of mixed models (linear, generalized linear, non-linear), with univariate as well as multivariate random effects, as long as the distribution for the outcomes conditional on the random effects is correctly specified. In case of model misspecification, the gradient function gives an important, albeit informal, indication on how the model can be improved in terms of random-effects distribution. The diagnostic value of the gradient function is extensively illustrated using some simulated examples, as well as in the analysis of a real longitudinal study with binary outcome values.

A Graphical Approach to Diagnosing the Validity of the Conditional Independence Assumptions of a Bayesian Network Given Data

Bayesian networks (BNs) have attained widespread use in data analysis and decision making. Well-studied topics include efficient inference, evidence propagation, parameter learning from data for complete and incomplete data scenarios, expert elicitation for calibrating BN probabilities, and structure learning. It is common for the researcher to assume the structure of the BN or to glean the structure from expert elicitation or domain knowledge. In this scenario, the model may be calibrated through learning the parameters from relevant data. There is a lack of work on model diagnostics for fitted BNs; this is the contribution of this article. We key on the definition of (conditional) independence to develop a graphical diagnostic that indicates whether the conditional independence assumptions imposed, when one assumes the structure of the BN, are supported by the data. We develop the approach theoretically and describe a Monte Carlo method to generate uncertainty measures for the consistency of the data with conditional independence assumptions under the model structure. We describe how this theoretical information and the data are presented in a graphical diagnostic tool. We demonstrate the approach through data simulated from BNs under different conditional independence assumptions. We also apply the diagnostic to a real-world dataset. The results presented in this article show that this approach is most feasible for smaller BNs—this is not peculiar to the proposed diagnostic graphic, but rather is related to the general difficulty of combining large BNs with data in any manner (such as through parameter estimation). It is the authors’ hope that this article helps highlight the need for more research into BN model diagnostics. This article has supplementary materials online.

Semiparametric regression in testicular germ cell data

It is possible to approach regression analysis with random covariates from a semiparametric perspective where information is combined from multiple multivariate sources. The approach assumes a semiparametric density ratio model where multivariate distributions are "regressed" on a reference distribution. A kernel density estimator can be constructed from many data sources in conjunction with the semiparametric model. The estimator is shown to be more efficient than the traditional single-sample kernel density estimator, and its optimal bandwidth is discussed in some detail. Each multivariate distribution and the corresponding conditional expectation (regression) of interest are estimated from the combined data using all sources. Graphical and quantitative diagnostic tools are suggested to assess model validity. The method is applied in quantifying the effect of height and age on weight of germ cell testicular cancer patients. Comparisons are made with multiple regression, generalized additive models (GAM) and nonparametric kernel regression.

Quantifying, displaying and accounting for heterogeneity in the meta-analysis of RCTs using standard and generalised Q statistics.

BackgroundClinical researchers have often preferred to use a fixed effects model for the primary interpretation of a meta-analysis. Heterogeneity is usually assessed via the well known Q and I2 statistics, along with the random effects estimate they imply. In recent years, alternative methods for quantifying heterogeneity have been proposed, that are based on a 'generalised' Q statistic.MethodsWe review 18 IPD meta-analyses of RCTs into treatments for cancer, in order to quantify the amount of heterogeneity present and also to discuss practical methods for explaining heterogeneity.ResultsDiffering results were obtained when the standard Q and I2 statistics were used to test for the presence of heterogeneity. The two meta-analyses with the largest amount of heterogeneity were investigated further, and on inspection the straightforward application of a random effects model was not deemed appropriate. Compared to the standard Q statistic, the generalised Q statistic provided a more accurate platform for estimating the amount of heterogeneity in the 18 meta-analyses.ConclusionsExplaining heterogeneity via the pre-specification of trial subgroups, graphical diagnostic tools and sensitivity analyses produced a more desirable outcome than an automatic application of the random effects model. Generalised Q statistic methods for quantifying and adjusting for heterogeneity should be incorporated as standard into statistical software. Software is provided to help achieve this aim.

MATLAB library LIBRA

AbstractLIBRA stands for ‘library for robust analysis.’ It is a MATLAB toolbox mainly containing implementations of robust statistical methods. Robust statistics is involved with the detection of aberrant observations, also called outliers. The aim of robust statistical methods is to provide estimates which are not affected by the nonregular observations and which are then able to pinpoint the outliers. The robust methods implemented in LIBRA nowadays cover: univariate location, scale (Qn) and skewness estimation (Medcouple), covariance estimation (FAST MCD), regression (FAST‐LTS, MCD regression, depth quantiles), principal component analysis (RAPCA, ROBPCA), principal component regression (RPCR), partial least squares regression (RSIMPLS), classification (RDA, RSIMCA) and several methods to deal with skewed data. Besides that, the toolbox contains various methods for cluster analysis, many graphical diagnostic tools, and classical equivalents of several implemented methods. Only a few of the recently added methods will be highlighted in this paper. The features of the LIBRA functions will be illustrated by means of some real data sets. Copyright © 2010 John Wiley & Sons, Inc.This article is categorized under: Software for Computational Statistics > Software/Statistical Software

DATA‐DRIVEN SMOOTH TESTS AND A DIAGNOSTIC TOOL FOR LACK‐OF‐FIT FOR CIRCULAR DATA

SummaryTwo contributions to the statistical analysis of circular data are given. First we construct data‐driven smooth goodness‐of‐fit tests for the circular von Mises assumption. As a second method, we propose a new graphical diagnostic tool for the detection of lack‐of‐fit for circular distributions. We illustrate our methods on two real datasets.