Abstract 2388: Artificial neural network based ensemble models for single-arm glioblastoma trials: A new historical control adjustment method

Abstract

Abstract Background: Despite a long history of poor decision making leading to repeated failed phase 3 trials, phase 2 single-arm studies continue to be common in glioblastoma (GBM). With the drawback of inflated false positive rate in current adjustment method for historical control data as concurrent controls, the new statistical methods and validation approach will be needed for adequate covariate adjustment in the GBM single-arm trial data. Methods: The systematic review was performed to extract the summary-level external data of survival outcomes (mean progression-free survival, mPFS, and PFS at 6 months, PFS6) and prognostic factors (age, gender, extent of resection, KPS, methylation status, and treatment history) from published randomized clinical trials for the standard of care (SOC) treatment in GBM. Four models of the standard, bootstrap aggregating, random subspaces, and random patches models were developed to construct a powerful predictive tool by comprising a series of connections producing survival outputs as a function of prognostic inputs using (1) artificial neural network (ANN) method and (2) linear method (Fig1). The root mean square errors (RMSEs) were calculated to evaluate the model performance using the five-fold cross-validation approach with the effects of ensemble size. The predictive survival outcomes among prognostic factors were calculated to evaluate the relative importance of factors. Results: We identified 19 Phase 2 and 3 clinical trials in glioblastoma and two survival outcomes and seven prognostic factors (Table 1). ANN models have less prediction error and therefore better performance on the test set than linear models. Presumably, the relationships in the data are highly nonlinear, and thus linear models cannot capture these relationships (Table 2). Bagging models typically outperform their standard counterparts for both ANNs and linear models. Ensemble methods can capture many trends within the data by manipulation of the training set and are therefore less prone to biases of individual models exacerbated by the small sample size. The previous treatment history and methylation status were the top two influential prognostic factors for both mPFS and PFS6, followed by the extent of resection and mean age (Fig2). With the examination of the ANN parameters of sample size, learning rate, training hyperparameters, it could be observed that there is no clear change in error when increasing sample size beyond 10 (Fig3). Conclusions: High failure rate of phase 3 glioblastoma trials emphasizes the need for greater reliability of earlier studies in phase 2 single-arm trials. The proposed statistical and epidemiologic approach based on the ensemble-based ANN models will be an effective tool to lead to better decision with phase 2 single-arm trials, which may give better chance to appropriately advance to phase 3 trials with higher predictive power. Citation Format: Wonsuk Yoo, Tortsen Ullrich, Nader Sanai. Artificial neural network based ensemble models for single-arm glioblastoma trials: A new historical control adjustment method [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2024; Part 1 (Regular Abstracts); 2024 Apr 5-10; San Diego, CA. Philadelphia (PA): AACR; Cancer Res 2024;84(6_Suppl):Abstract nr 2388.