Abstract 110: Planned versus observed effect sizes in early phase chemoprevention trials

Abstract

Abstract Early phase chemoprevention trials are designed to demonstrate safety, tolerability, feasibility, and signals of efficacy. Yet it is often observed that most trials fail to detect intervention effects. The purpose of this review was to analyze intervention effects in such trials by comparing planned vs observed effect sizes. Single or multi-arm efficacy and biomarker chemoprevention trials conducted under the Phase 0-II Cancer Prevention Clinical Trial Program of the Division of Cancer Prevention, National Cancer Institute between 2003 and 2019 were evaluated. Protocols were reviewed to gather information regarding study design, primary endpoint(s), analysis plan, analysis population(s), and sample size justifications with planned effect sizes. Reports and manuscripts were reviewed to obtain study results information. Intervention effects for differences were quantified by calculating standardized Cohen's d effect sizes. In studies where only p-values were reported without effect sizes, observed effect sizes were estimated using p-values and sample size information. Planned effect sizes were compared to observed effect sizes and post-hoc power calculations were conducted. Of a total of 59 trials reviewed, twenty-four trials were efficacy or biomarker studies with complete information on planned and observed effect sizes. Organ sites of focus included 5 (21%) breast, 6 (25%) prostate, 4 (17%) lung, and 9 (38%) gastrointestinal and other sites. The majority of the trials (n=18) were multi-arm randomized studies of which 15 trials were blinded. The median planned effect size for detecting an intervention effect was 0.75 (range 0.2-1.14), while the median planned sample size per arm was n=30 (range 11-101). The planned effect sizes were larger than the observed effects sizes in the majority of the studies (88%). The observed effect sizes in most studies were small with a median of 0.34 (range 0-0.66). The median difference between planned vs observed effect size was -0.37. A low level of concordance between planned vs observed effect size was observed with an intra-class correlation coefficient 0.18 (95% CI: -0.1-0.50). The median post-hoc power for detecting the observed effect sizes was only 0.29 (range 0.03-0.93). There were no significant differences detected between planned vs observed effect sizes when stratified by accrual goals, analysis populations, or study design (multi-arm randomized vs single arm). The results indicate that the majority of early phase chemoprevention trials have much smaller observed effect sizes than planned. Sample size calculations for such trials need to balance the known attributes of the study endpoints, e.g. the potential detectable effect sizes in populations that can be realistically and cost-effectively accrued, with the need to detect only effect sizes large enough to justify subsequent phase III trials. Utilization of interim analyses for futility may be considered. Citation Format: Jens C. Eickhoff, Jen Birstler, Guanhua Chen, Zhumin Zhang, Vikrant Sahasrabuddhe, Eva Szabo, KyungMann Kim. Planned versus observed effect sizes in early phase chemoprevention trials [abstract]. In: Proceedings of the American Association for Cancer Research Annual Meeting 2021; 2021 Apr 10-15 and May 17-21. Philadelphia (PA): AACR; Cancer Res 2021;81(13_Suppl):Abstract nr 110.