The LAS and Novel Models Poorly Predict Post-Transplant Survival

Abstract

<h3>Purpose</h3> The Lung Allocation Score (LAS) was designed to allocate donor lungs based on estimates of waitlist mortality and post-transplant survival and has increased the number of lung transplants performed and reduced waitlist mortality. As a component of net "transplant benefit", our aim was to compare the ability of the LAS and 3 novel methods to predict 1-year post-transplant death or retransplant. We also evaluated methods to improve the predictive validity of these models. <h3>Methods</h3> Utilizing the Organ Procurement and Transportation Network database (2005-2017), we compared prediction of post-transplant survival of the LAS, a model described by Chan, <i>et al.</i>, a logistic regression model utilizing <i>a priori</i> expert selection of pre-transplant recipient covariates, and two machine learning models, Least Absolute Shrinkage and Selection Operator (LASSO) and Random Forests, using all available covariates. We compared the area under the receiver-operator characteristic curve (AUC), positive predictive value (PPV), and negative predictive value (NPV) between the models. We evaluated the calibration of each model by comparing average predicted probability of outcomes in each decile to observed outcome percentages. We repeated the analysis evaluating 3-year post-transplant mortality, fitting separate models to disease category, and including donor covariates. <h3>Results</h3> All models demonstrated low AUC (0.55-0.63), poor PPV (<0.23), but relatively high NPV (0.87-0.9). The Random Forests and <i>a priori</i> expert models had higher AUC (both 0.63) than the LAS (0.55), LASSO (0.61), and Chan, <i>et al.</i> (0.59) models. Evaluating calibration of the LAS found that 1-year post-transplant estimates from the LAS consistently overestimated the risk of post-transplant mortality, with greater differences in higher LAS deciles. The LASSO, Random Forests, and <i>a priori</i> expert models showed no improvement in AUC when each were fit to individual disease groups or evaluated with the addition of donor covariates and had lower AUC when evaluated for 3-year outcomes. <h3>Conclusion</h3> The LAS can overestimate a patient's risk of post-transplant death and thus underestimate transplant benefit in the sickest candidates. Novel models based on pre-transplant recipient covariates fail to improve prediction. There should be wariness in the short-term post-transplant survival predictions from the LAS.