Individualized Treatment Effect Prediction with Machine Learning — Salient Considerations

Abstract

BackgroundMachine learning–based approaches that seek to accomplish individualized treatment effect prediction have gained traction; however, some salient challenges lack wider recognition.MethodsWe describe key methodologic considerations for individualized treatment effect prediction models using data from the Treatment of Preserved Cardiac Function Heart Failure with an Aldosterone Antagonist Trial for spironolactone in heart failure with preserved ejection fraction. The causal survival forest algorithm was used for model development. Calibration and discrimination were evaluated using a bootstrapping-based internal validation procedure. Observed benefits were described for predicted benefit quartiles and quartiles of a known effect modifier: ejection fraction. A negative control analysis with noncardiovascular death as the outcome was implemented to detect confounding.ResultsAmong 3445 participants, 671 events occurred over a median of 3.3 years of follow-up. In internal validation, a higher average observed benefit was noted among patients in the highest quartile of predicted benefit. The median (interquartile range) of the observed restricted mean survival time difference at 3.3 years at the highest quartile of model-predicted benefit was 62 days (32 to 83) and was 47 days (26 to 67) at the lowest quartile of ejection fraction. Body-mass index had higher contribution to prediction of benefit relative to other included measures (33.7% vs. glomerular filtration rate [27.3%], ejection fraction [15.1%], and younger age [12.8%]) No benefit was observed for noncardiovascular death at higher model-predicted benefit quartiles, although benefit for noncardiovascular death was observed at lower quartiles.ConclusionsCarefully applied and validated predictive models hold promise in identifying heterogeneous treatment effects and are useful for hypothesis generation regarding the role of phenotypic characteristics in modifying the benefit of experimental interventions in clinical trials. (Funded by the National Heart, Lung, and Blood Institute; ClinicalTrials.gov number, NCT00094302.)