Abstract 34: Improve decision making in clinical trials through machine learning and EHR

Abstract

Abstract One of the key contributors to the failure of oncology trials is patient selection bias. A bias that results in a lower observed efficacy in phase II may lead to a premature decision to abort a trial, whereas a bias in the positive direction may result in an expensive failure in phase III. We propose an approach in which RWE is used to model a proxy endpoint that is available in RWE but is also related to the endpoint in clinical trial, on patient profile; the model is then used on clinical trial patients to predict their proxy endpoints, which in turn is used to quantitatively assess selection bias and infer the effect of bias on the observed efficacy of the trial. We used an ongoing metastatic breast cancer trial to demonstrate this approach. Medication, demography, vital signs, and time of death of the target patient population were extracted from Optum Humetica. These data were then used to train a ML survival model to predict OS based on patient profiles in a virtual trial. For this oncology trial, we chose OS as a proxy endpoint as it has a well-established strong correlation with the observed efficacy in terms of PFS and time on drug duration. OS is an endpoint. The term proxy is used here in the sense that it is not the endpoint used in the earlier-phase trial. The model is then used to predict risk profiles for patients in the actual trial, and the difference between risk profile of the trial patients and that of the RWE target population provides a quantitative assessment of selection bias of the trial population. The effectiveness of this approach depends on the predictive power of the model. A nonpredictive model obviously cannot detect bias even when it is present. To quantitatively assess the predictive power of the model, we subsampled the RWE population to simulate a trial population with different risk profiles and use the model to determine a threshold in risk difference beyond which the model can detect a significant difference. In this context, we also compared Cox PH, decision tree, and a neural network model with respect to their predictive power in term of concordance score. We then compared the risk profile of the clinical trial patients with that of the target patient population. As RWE data become more abundant and data interoperability and quality improve, the prediction model will become more accurate and bias detection will become more sensitive. We believe this approach will lead to an effective framework whereby RWE data can be used to improve the success rate of oncology clinical trials. Citation Format: LiMing Shen, Tao Sheng, Xaiodong Luo. Improve decision making in clinical trials through machine learning and EHR [abstract]. In: Proceedings of the AACR Special Conference on Advancing Precision Medicine Drug Development: Incorporation of Real-World Data and Other Novel Strategies; Jan 9-12, 2020; San Diego, CA. Philadelphia (PA): AACR; Clin Cancer Res 2020;26(12_Suppl_1):Abstract nr 34.