Response to Kalbfleish and Schaubel: “A Perspective on the Scientific Registry of Transplant Recipients Migration to Bayesian Methods”

Abstract

To the Editor: Kalbfleisch and Schaubel make several interesting points 1. Although using the one-sided p-value is an optimal frequentist hypothesis test criterion, it is not necessarily the optimal way to construct a screening test for regulatory review of posttransplant outcomes. First, the optimal criterion for a screening test depends on the trade-off between true and false positives. Failing to identify truly underperforming programs for review can have negative consequences for their patients and prevent making the best use of scarce donated organs. Unnecessarily reviewing programs that are not truly underperforming wastes effort and can have additional negative consequences. The Scientific Registry of Transplant Recipients (SRTR), with input from the Organ Procurement and Transplantation Network's Membership and Professional Standards Committee (MPSC), attempted to balance these costs when developing screening criteria. Although the selected screening criteria provide similar results to screening with a one-sided p-value, this similarity is coincidental. Second, hypothesis test procedures do not always achieve their claimed significance level. This often occurs when the data can only have discrete values, which is the case for counts of graft failures or deaths. Since we can never have a non-integer number of events, a screening test that identified programs when the one-sided p-value is less than 0.05 would often have a Type I error rate less than 0.05. By considering the screening test tradeoffs, the MPSC's new criteria allow the false positive rate to sometimes exceed 5% if it produces sufficient improvement in the true positive rate 2. The Committee of Presidents of Statistical Societies report focuses on empirical priors and hierarchical models rather than on the Bayesian framework that SRTR chose, but the report unambiguously promotes statistical shrinkage, which is consistent with switching to a Bayesian framework 3. It is likely that the true distribution of program hazard ratios is difficult to summarize with a simple prior distribution. The prior that is most consistent with national data has a lower variance than SRTR's prior, but that doesn't mean that data from all programs are consistent with a lower-variance prior. Table 1 shows data from four unusual programs whose observed data are highly unlikely under a low-variance prior, but are notably more likely under the SRTR prior. The empirical priors may better describe the majority of transplant programs, but extreme data are much more common than those priors imply. There are reasonable arguments favoring lower-variance and higher-variance priors. SRTR considers its current prior an acceptable compromise for reasons discussed in the original paper 4. N. Salkowski1, J. J. Snyder1,2 and B. L. Kasiske1,3,* 1Scientific Registry of Transplant Recipients Minneapolis Medical Research Foundation Minneapolis, MN 2Division of Epidemiology and Community Health School of Public Health, University of Minnesota Minneapolis, MN 3Department of Medicine, Hennepin County Medical Center Minneapolis, MN *Corresponding author: Bertram L. Kasiske, Bkasiske@cdrg.orgBkasiske@cdrg.org The authors of this manuscript have no conflicts of interest to disclose as described by the American Journal of Transplantation.