A Critique on Contemporary Reporting of Likelihood Ratios in Test Power Analysis

Abstract

In the current issue of the Mayo Clinic Proceedings, an exchange of letters to the editor relating to the selection of the appropriate measures of the power of clinical indicators (ie, clinical symptoms or signs) and laboratory tests for the diagnosis of giant cell arteritis (GCA) epitomizes a continuing dilemma among clinicians in the interpretation of sensitivity (Se), specificity (Sp), and likelihood ratios (LRs). In his letter to the editor, McGee1McGee S Calculating likelihood ratios in patients with giant cell arteritis [letter].Mayo Clin Proc. 2004; 79: 1341Abstract Full Text Full Text PDF PubMed Scopus (2) Google Scholar criticizes Younge et al2Younge BR Cook Jr, BE Bartley GB Hodge DO Hunder GG Initiation of glucocorticoid therapy: before or after temporal artery biopsy? [published correction appears in Mayo Clin Proc. 2004;79:709].Mayo Clin Proc. 2004; 79: 483-491Abstract Full Text Full Text PDF PubMed Scopus (91) Google Scholar in their reporting of a “novel definition of negative likelihood ratio” as “specificity/1 – sensitivity.” McGee recommends the computation of the negative LR in the form “(1 – sensitivity)/specificity,” which is the inverse of the negative LR reported by Younge et al. The reporting of Se, Sp, and LRs is based on the mathematical principle of conditional probability, which was first enunciated in 1763 in the now famous essay by Reverend Thomas Bayes.3Bayes T An essay towards solving a problem in the doctrine of chances.Philos Trans. 1763; 53: 370-418Crossref Google Scholar By this principle, the power to predict an event, such as disease (in this article, the term disease refers generically to clinical disease, dysfunction, disorder, or demise), on the basis of its association with a given condition, such as a positive test result, is derived as the frequency of the condition in the presence of the event divided by the frequency of the same condition in the absence of the event. By the same principle, the power to predict the absence of an event, such as absence of disease, on the basis of the absence of a given condition, such as a negative test result, is derived as the frequency of the absence of the condition in the absence of the event divided by the frequency of the absence of the same condition in the presence of the event. These quotients are referred to as Bayes factors in the current literature on Bayesian statistical analysis.4Good IJ A Bayesian significance test for multinomial distributions.J R Stat Soc B. 1967; 29: 399-431Google Scholar, 5Bernardo JM Smith AFM Bayesian Theory. Wiley, Chichester, England1994: 389-394Google Scholar, 6Kass RE Raftery AE Bayes factors.J Am Stat Assoc. 1995; 90: 773-795Crossref Scopus (11038) Google Scholar When the Bayes factors are derived from simple single distributions such as positive or negative results, the quotients are termed likelihood ratios.6Kass RE Raftery AE Bayes factors.J Am Stat Assoc. 1995; 90: 773-795Crossref Scopus (11038) Google Scholar The LRs of a clinical indicator or laboratory test reported in the current literature are derived from measures of Se, Sp, and their complements as follows: (1) the LR for disease for a positive result, which we denote by (+)LRd, calculated as [Se/(1 – Sp)]; (2) the LR for no disease (normalcy) for a negative result, which we denote by (–)LRn, calculated as [Sp/(1 – Se)]; and (3) the LR for disease for a negative result, which we denote by (–)LRd (conventionally abbreviated as LR–), calculated as [(1 – Se)/Sp], ie, the inverse of (2). A fourth potential measure, that of the LR for no disease for a positive result, ie, the inverse of (1), is rarely, if ever, reported and will not be discussed. Each of these ratios constitutes an odds ratio measure of the power of a positive or negative test result to predict disease or no disease, relative to the corresponding odds of disease or no disease in the overall population under study. The first ratio, (+)LRd, is directly related to the power of a positive test result to predict disease. The second ratio, (–)LRn, is directly related to the power of a negative test result to predict the absence of disease. The third ratio, (–)LRd, is directly related to the power of a negative test result to predict disease. Because clinicians do not associate a normal or negative finding with the prediction of disease, the reporting of this third ratio is counterintuitive to clinical practice. Nevertheless, in the current literature, it is the LRs in the form of the (+)LRd and (–)LRd that are the most frequently reported, to the exclusion of the (–)LRn. The major limitations we find in reporting predictive power using measures (1) and (3) are the ambiguity in the comparison of positive with negative predictive power by these LRs and the focusing of attention on the power to predict disease without attention to the power to predict the absence of disease. Thus, in interpreting predictive power in terms of (+)LRd and (–)LRd, the higher the power of a test in terms of its Se and Sp, the higher the (+)LRd will be and the lower its (–)LRd will be. Conversely, the lower the Se and Sp of a test, the lower the (+)LRd will be and the higher the (–)LRd will be. With regard to the estimation of the negative predictive power of a test, when test power analysis is confined to the comparison of (+)LRd and (–)LRd, the analysis excludes a direct measure of the power of a clinical indicator or test to predict the absence of disease for a negative result. Therefore, this conventional practice omits the clinically relevant comparison of the rule-in power of a positive result with the rule-out power of a negative result on a common scale of probability measures.7Weissler AM A perspective on standardizing the predictive power of noninvasive cardiovascular tests by likelihood ratio computation: 1. Mathematical principles.Mayo Clin Proc. 1999; 74: 1061-1071Abstract Full Text Full Text PDF PubMed Scopus (25) Google Scholar, 8Weissler AM Use of likelihood ratio computation to standardize the predictive power of noninvasive cardiovascular tests [reply].Mayo Clin Proc. 2000; 75: 423-424Abstract Full Text Full Text PDF PubMed Google Scholar A comparison of the 3 published LR options in test power evaluation is illustrated in Table 1. The data are drawn from Table 3 in the article by Younge et al.2Younge BR Cook Jr, BE Bartley GB Hodge DO Hunder GG Initiation of glucocorticoid therapy: before or after temporal artery biopsy? [published correction appears in Mayo Clin Proc. 2004;79:709].Mayo Clin Proc. 2004; 79: 483-491Abstract Full Text Full Text PDF PubMed Scopus (91) Google Scholar The variables selected for this comparison are the single symptomatic indicators and laboratory tests for GCA among 1113 patients who underwent temporal artery biopsy. In view of the key influence of corticosteroid therapy, which normalizes the elevated erythrocyte sedimentation rate (ESR) in this disease, the LRs for ESR are listed for the subpopulation that did not receive corticosteroid therapy before testing (labeled incorrectly in the original table as “Normal ESR”) and for the whole population. The data are listed in descending order of the rule-out power for a negative test result as measured by the (–)LRn.TABLE 1Comparison of 3 Likelihood Ratio (LR) Measures of the Predictive Power of Clinical Indicators and Laboratory Tests in the Diagnosis of Giant Cell Arteritis (GCA)*ESR = erythrocyte sedimentation rate.Data from Younge et al.2Younge BR Cook Jr, BE Bartley GB Hodge DO Hunder GG Initiation of glucocorticoid therapy: before or after temporal artery biopsy? [published correction appears in Mayo Clin Proc. 2004;79:709].Mayo Clin Proc. 2004; 79: 483-491Abstract Full Text Full Text PDF PubMed Scopus (91) Google ScholarVariablePositive LR for GCA [(+)LRd]Negative LR for no GCA [(–)LRn]Negative LR for GCA [(–)LRd]ESR†Among patients not taking corticosteroids.1.240.00.025ESR‡Among all patients.1.23.60.28New headache1.71.80.56Jaw claudication6.71.60.64Scalp tenderness3.01.30.75Abnormal hemoglobin1.11.30.79Abnormal platelet count1.51.20.86Weight loss1.31.10.93Decreased vision1.61.10.94Double vision3.51.00.97* ESR = erythrocyte sedimentation rate.† Among patients not taking corticosteroids.‡ Among all patients. Open table in a new tab The data in the first 2 LR columns allow ready comparison of the positive predictive power of a clinical indicator or laboratory test to rule in GCA by the (+)LRd with the negative predictive power to rule out GCA by the (–)LRn. The marked difference between the high rule-out power of a normal ESR among patients who did not receive corticosteroids [(–)LRn of 40.0] and the low rule-in power of an abnormal ESR for GCA [(+)LRd of 1.2] in patients who were and those who were not receiving corticosteroid therapy before temporal artery biopsy is evident. The higher rule-in power for GCA of the symptoms of jaw claudication, scalp tenderness, and double vision as estimated by the (+)LRd than the rule-out power of these same symptoms as estimated by the (–)LRn is clearly apparent. To make these same comparisons using (–)LRd, the reader must mentally invert ratio (3), a maneuver that unnecessarily complicates the analysis. On the basis of the ambiguity introduced in predictive power analysis by the combined reporting of the (+)LRd with the (–)LRd, and the omission of a distinct measure of power of a negative test to predict absence of disease, we recommend that the presentation of LR data be confined to the measures of (+)LRd and (–)LRn. Furthermore, we recommend that the reporting of these ratios include the augmented notation of LR measures in which both the conditions of prediction and the event to be predicted are specified. Calculating Likelihood Ratios in Patients With Giant Cell ArteritisMayo Clinic ProceedingsVol. 79Issue 10PreviewTo the Editor: Younge et al1 provide a landmark study that will allow clinicians to determine accurately the probability of giant cell arteritis in patients before temporal artery biopsy. Nonetheless, Table 3 in their article is confusing because it uses a novel definition of negative likelihood ratio (LR). Likelihood ratios are defined conventionally as the probability of a particular finding in patients with disease divided by the probability of the same finding in patients without disease.2 An LR is “positive” when it refers to the presence of the finding; it is “negative” when it refers to the absence of the finding. Full-Text PDF Calculating Likelihood Ratios in Patients With Giant Cell Arteritis: In ResponseMayo Clinic ProceedingsVol. 79Issue 10PreviewWe thank Dr McGee for his letter and for his interest in our study on temporal artery biopsies.1 As he noted, LRs compare the probability of a particular positive or negative test result in patients with a disease to the probability of the test result in those without the disease. As we described in the “Patients and Methods” section of our article, we chose to invert the formula for calculating negative LRs. We believed that this would make comparisons of positive and negative LRs easier, as suggested by Feinstein2 and Weissler. Full-Text PDF