Refining the Lung Allocation Score Models Fails to Improve Discrimination Performance

Abstract

As broader geographic sharing is implemented in lung transplant allocation through the Composite Allocation Score (CAS) system, models predicting waitlist and posttransplant (PT) survival will become more important in determining access to organs. How well do CAS survival models perform, and can discrimination performance be improved with alternative statistical models or machine learning approaches? Scientific Registry for Transplant Recipients (SRTR) data from 2015-2020 were used to build seven waitlist (WL) and data from 2010-2020 to build similar PT models. These included the (I) current lung allocation score (LAS)/CAS model; (II) re-estimated WL-LAS/CAS model; (III) model II incorporating nonlinear relationships; (IV) random survival forests model; (V) logistic model; (VI) linear discriminant analysis; and (VII) gradient-boosted tree model. Discrimination performance was evaluated at 1, 3, and 6months on the waiting list and 1, 3, and 5 years PT. Area under the curve (AUC) values were estimated across subgroups. WL model performance was similar across models with the greatest discrimination in the baseline cohort (AUC 0.93) and declined to 0.87-0.89 for 3-month and 0.84-0.85 for 6-month predictions and further diminished for residual cohorts. Discrimination performance for PT models ranged from AUC 0.58-0.61 and remained stable with increasing forecasting times but was slightly worse for residual cohorts. WL and PT variability in AUC was greatest for individuals with Medicaid insurance. Use of alternative modeling strategies and contemporary cohorts did not improve performance of models determining access to lung transplant.