Miscalibration of lung allocation models leads to inaccurate waitlist mortality predictions

Abstract

The importance of waitlist (WL) mortality risk estimates will increase with the adoption of the US Composite Allocation Score (CAS) system. Calibration is rarely assessed in clinical prediction models, yet it is a key factor in determining access to lung transplant. We assessed the calibration of the WL–lung allocation score (LAS)/CAS models and developed alternative models to minimize miscalibration. Scientific Registry of Transplant Recipients data from 2015 to 2020 were used to assess the calibration of the WL model and for subgroups (age, sex, diagnosis, and race/ethnicity). Three recalibrated models were developed and compared: (1) simple recalibration model (SRM), (2) weighted recalibration model 1 (WRM1), and (3) weighted recalibration model 2 (WRM2). The current WL-LAS/CAS model underestimated risk for 78% of individuals (predicted mortality risk, <42%) and overpredicted risk for 22% of individuals (predicted mortality risk, ≥42%), with divergent results among subgroups. Error measures improved in SRM, WRM1, and WRM2. SRM generally preserved candidate rankings, whereas WRM1 and WRM2 led to changes in ranking by age and diagnosis. Differential miscalibration occurred in the WL-LAS/CAS model, which improved with recalibration measures. Further inquiry is needed to develop mortality models in which risk predictions approximate observed data to ensure accurate ranking and timely access to transplant. ImpactWith changes to the lung transplant allocation system planned in 2023, evaluation of the accuracy and precision of survival models used to rank candidates for lung transplant is important. The waitlist model underpredicts risk for 78% of US transplant candidates with an unequal distribution of miscalibration across subgroups leading to inaccurate ranking of transplant candidates. This work will serve to inform future efforts to improve modeling efforts in the US lung transplant allocation system.