With or Without a Gold Standard

Abstract

To explore causal relationships between exposure and outcome, epidemiologists must rely on accurate measurements of both. Misclassification of either exposure or outcome will obscure causality, ie, by an inability to distinguish exposed from unexposed or “diseased” from “nondiseased.” Whenever new and purportedly better (but nonvalidated) measurements of exposure or outcome become available, epidemiologists are faced with a fundamental question of “Which test is better?” Tools for answering this question differ depending on whether a true gold standard is available. In this issue of EPIDEMIOLOGY, articles by Pepe 1 and Hagdu 2 address some of the issues surrounding comparisons of diagnostic tests with 1 and without 2 a gold standard. The receiver-operator characteristics curve (ROC) is a popular summary measure of the discriminatory ability of a clinical marker that can be used when there is a gold standard. The ROC plots sensitivity against 1-specificity (true versus false positivity) for all thresholds that could have been used to define “test positive.” An ROC curve is assessed by measuring the area under the curve (AUC), which ranges from 0.5 (no discriminatory ability) to 1.0 (perfect discriminatory ability). Two diagnostic tests can be compared by calculating the difference between the areas under their 2 ROC curves. Pepe 1 offers an alternative (nonparametric) presentation of ROC analysis that may make such analysis more accessible to epidemiologic researchers. Pepe relates the ROC curve to standardized values of the clinical marker, termed “placement values.” The placement value for an affected individual with maker value Y is the proportion of unaffected subjects (the reference population) with values higher than Y. To extend this idea to the whole population, the mean placement value or average of the percentiles is computed by averaging the placement values of the affected subjects, with the mean placement value reflecting the distribution of values in the control population. A low mean placement value indicates that the clinical marker distinguishes patients with an outcome from those without an outcome in that population. Using the fact that the ROC curve is the probability distribution of the placement values, Pepe demonstrates that the mean placement value is equal to 1-AUC. A nice feature of using placement values is that they are amenable to regression analysis to identify and control for factors that modify the performance of the clinical measurement. Both of these related approaches use all thresholds for the definition of “test positive.” In practice, users often confine their attention to regions of the ROC curve that correspond to the most clinically relevant values of test sensitivity or specificity. For example, a test that performs well for low specificity but poorly for high specificity may not be desirable. In this case, the partial area under the curve is a more appropriate measure of test performance than the AUC.