Machine learning prediction of progression events in oncology RECIST 1.1 clinical trials.

Abstract

e13589 Background: Oncology clinical trials often have an imaging based surrogate endpoint such as Progression Free Survival (PFS). Using PFS as a surrogate for survival has the advantage of achieving trial completion more quickly which reduces cost and risk to patients. These trials are frequently structured to require a certain number of events to achieve statistical significance. The use of machine learning models which can make predictions about progression events from data obtained after the initiation of therapy is a novel idea in this field. It is well understood that many factors play a role in whether a patient will ultimately progress on a certain therapy. While certain statistical predictions can be made given known factors such as tumor genetics and patient history, these are made prior to the patient initiating therapy and do not consider on-therapy response. Machine learning models can be used during trial conduct, incorporate patient response, and better predict progression outcomes. Methods: Machine learning models were trained on over 115,000 RECIST 1.1 datapoints. The iterative training process included inputs from the medical science team which included a lead radiologist with expertise in diagnostic radiology and molecular imaging; and a lead medical informaticist responsible for data quality. Models were trained using 56 raw and derived features. Examples of derived features include number of lesions, time between evaluations, and change in responses over time. The features were trained using Random Forest, AdaBoost, Gradient Boost, XGBoost model types and compared to determine which one performed best. These model types identified the top predictors and created tree-based visuals, which provided insights into the model’s decision-making process. Confusion matrices were reviewed by the team to iteratively address false positive and false negative records. Results: Initial results of the machine learning model demonstrated a 75.1% specificity and a 66.6% accuracy for prediction of progression events. With continuous improvement efforts, as of writing, the model demonstrated 94.4% specificity, and 82.2% accuracy. This process identified the top predictors of progression to be the primary RECIST 1.1 features. Given our scientific and AI expertise results are expected to improve over time. Conclusions: Outcomes have shown the ability to predict RECIST 1.1 progression. Such a model could successfully predict when the required number of progression events are present within a population, at which point enrollment could be ceased and decrease overall trial length. Given enough data and rigorous testing, this technology could predict complete response vs partial response, adverse events, and time to events. Data suggests that AI machine learning within Clinical Trials could have a significant impact and should be considered a priority for continued evaluation given the possible benefits to trials.