Rebalancing Ratios and Improving Impressions: Later Thoughts From the Prostate Cancer Prevention Trial Investigators

Abstract

By defining type I and type II errors of inference, the great biostatisticians Neyman and Pearson gave medical investigators both a powerful analytic tool and a crucial perspective for evaluating medical innovations. A type I error rejects the null hypothesis when it is true, whereas a type II error accepts it when it is not. By keeping these categories in mind, Neyman and Pearson correctly reasoned, investigators could explicitly weigh the consequences of mistaken conclusions in opposite directions. Depending on circumstances, one error or the other would likely cause greater harm. The cost of a type I error increases when the treated condition is less harmful; the study intervention is more toxic, invasive, or costly; and attractive alternatives are available. Conversely, type II errors weigh more heavily when the condition is dire, the intervention benign and inexpensive, and alternatives are unavailable or unappealing. After considering the clinical situation, investigators could tilt the study design or analysis plan away from the more damaging error, at the cost of increased risk of the more benign. Requiring stronger evidence that the null hypothesis is unlikely, by reducing the value or increasing the size of the clinically meaningful benefit the study is powered to detect, makes a type I error less likely, whereas greater study power, from increased enrollment or longer follow-up, reduces the chance of a type II error by producing a more precise estimate of the treatment effect. Although Neyman and Pearson’s approach applies most directly to decisions to reject or accept the null hypothesis, it is broadly relevant to choices made under uncertainty: if we make a mistake, it should be the least damaging. In their examination of factors that might alter the previously reported, conflicting results of the Prostate Cancer Prevention Trial (PCPT) in the current issue of the Journal of Clinical Oncology, Klein et al give Neyman and Pearson’s insight only glancing attention. Rather than examining the relative consequences of the opposing effects found in their path-breaking study, they instead focus on assumptions that would make using finasteride to prevent prostate cancer look more attractive than their originally published results suggested. To determine whether finasteride could prevent prostate cancer, the designers of the PCPT required a persuasive study outcome. Because of prostate cancer’s long natural history, the definitive and unambiguous outcome, mortality, which would require 15 or more years of follow-up, was out of practical reach. The study outcome they chose, the diagnosis of prostate cancer, posed difficulties because the tumor is so prevalent: it can be found in the prostates of more than half of men older than age 50. As a result, an encounter with a urologist and, subsequently, his biopsy gun is the most powerful risk factor for its diagnosis. This observation seems jokey, a tautologic observation that to diagnose this cancer—like many others— requires a biopsy and a surgeon to perform it. But those familiar with prostate cancer’s astonishing recent history and its intimate association with prostate-specific antigen (PSA) screening, which certainly include the PCPT investigators, understand its clinical importance. Practice patterns have changed the incidence of prostate cancer dramatically and suddenly. For example, in the month after the uncontrolled observation was prominently published that systematic prostate biopsies for a PSA level 4.0 ng/mL frequently detect prostate cancers in asymptomatic men, prostate cancer diagnoses increased 87% in men in the Health Professionals Follow-Up Study and continued to increase steeply from the newly elevated baseline. Given that the goal of finasteride therapy in the PCPT was not to inhibit the diagnosis of prostate cancer but rather JOURNAL OF CLINICAL ONCOLOGY E D I T O R I A L VOLUME 23 NUMBER 30 OCTOBER 2