Development and Validation of a Machine-Learning Model to Predict POD24 Risk of Follicular Lymphoma

Abstract

Background: Disease progression or relapse within 24 months after starting treatment (POD24) has been considered as an independent unfavorable factor in follicular lymphoma (FL). This study was to explore the discriminative accuracy of a machine learning-based (ML) model within different ethnic groups for identifying FL1-3a patients at higher risk for POD24. Methods: 1938 FL1-3a patients were enrolled from a Chinese multicenter cohort and randomly subdivided into a training cohort and an internal validation cohort. An external validation cohort included 1145 patients from the GALLIUM Study within different ethnicity. Univariable regression analysis with backward selection for inclusion of predictor variables and nonlinear analysis based on the XGBoost algorithm were used to develop a ML model for predicting POD24. For internal and external validation, the time-dependent area under the receiver operating characteristic curve (AUROC) was used to investigate the model's predictive performance, compared with established traditional models such as Follicular Lymphoma International Prognostic Index (FLIPI), FLIPI-2 and PRIMA-PI. The calibration and clinical usefulness of the ML model were evaluated using calibration plots and decision curve analyses, respectively. Results: During the follow-up period, 383 (19.7%) and 405 (36.3%) of patients who experienced POD24 were identified in the Chinese cohort and the GALLIUM Study, respectively. In the training cohort, important features of POD24 based on the XGBoost algorithm were ranked by SHAP analysis. Increased lymphocyte-to-monocyte ratio (LMR>10) ranked first (scoring 2), followed by elevated lactate dehydrogenase (LDH), hemogolobin reduction (HGB<12g/dl), elevated beta-2 microglobulin (B2-MG), higher maximum standardized uptake value (SUVmax>10), and 4 or more involved lymph nodes (each scoring 1) were incorporated into the new ML model, referred to as FLIPI-C. The new model performed well in predicting PFS as well as OS, and stratified patients into low- (0-3) and high-risk groups (4-7). In internal validation, FLIPI-C demonstrated a higher AUROC of 0.764 (95%CI:0.721-0.806) for POD24 prediction compared with 0.648 (95%CI: 0.599-0.696) of FLIPI, 0.706 (95%CI: 0.658-0.754) of FLIPI-2 and 0.716 (95%CI: 0.669-0.763) of PRIMA-PI. In external validation with GALLIUM Study, FLIPI-C demonstrated a higher AUROC of 0.701 (95%CI: 0.659-0.741) for POD24 prediction compared with 0.578 (95%CI: 0.531-0.625) of FLIPI, 0.600 (95%CI: 0.554-0.645) of FLIPI-2 and 0.593 (95%CI: 0.547-0.639) of PRIMA-PI. In addtion, the FLIPI-C model had adequate calibration with similar predicted and observed risk of POD24. In decision curve analysis, FLIPI-C yielded improved net benefits compared with FLIPI, FLIPI-2 and PRIMA-PI. Conclusions: TheFLIPI-C model generated using a machine learning approach exhibited greater discriminative accuracy than prior established traditional models for predicting POD24 and is valuable for treatment selection and prognostic assessment of FL.