Comparing phase 3 “go” decisions (Ph3-GO) between single arm trials with real-world external control (SAT+rwEC) versus randomized phase 2 trials (rPh2).

Abstract

e13564 Background: Traditional rPh2 trials have limitations that may yield suboptimal Ph3-GO. Compared to a rPh2 of equivalent sample size, SAT+rwEC allows more patients to receive experimental therapies while preserving the ability to compare experimental and control groups. Bias arising from measurement error and confounding in the rwEC, however, poses challenges to statistical inference. Preliminary studies suggest higher response rates are observed in rwEC than randomized controls. We compared Ph3-GO decisions between SAT+rwEC and rPh2. Methods: Ph3-GO probability was compared using simulation studies that resembled the oncology setting with objective response rate (ORR) endpoint. rPh2 simulated parameters were: sample size (60-120) with 1:1 randomization, ORR in rPh2 control (15%-50%), true treatment effect (ΔORR: 0-50). For each rPh2 of a given sample size, we evaluated an SAT+rwEC that re-allocated all rPh2 control patients to the experimental arm (i.e., doubling the sample size of the experimental arm) and added an rwEC. SAT+rwEC were simulated with assumptions for size (rwEC to SAT ratio: 0.5 to 2) and net bias (-10 to +10), which was simulated as a composite representing ORR measurement error plus residual confounding after multivariable adjustment. Positive direction of net bias corresponds to higher ORR in the rwEC. Ph3-GO thresholds varied from 10-30%. Ph3-GO was considered “False-GO” when true treatment effect < threshold, and “True-GO” when true treatment effect ≥ threshold. Results: With positive net bias of +10, SAT+rwEC had lower False-GO and True-GO decisions compared to rPh2. With negative net bias of -10, both False-GO and True-GO probabilities were higher for the SAT+rwEC. When net bias=0, the increased size of SAT+rwEC resulted in observable Ph3-GO improvements with lower False-GO and higher True-GO than corresponding rPh2. Conclusions: An interactive dashboard was developed for users. The magnitude and direction of net bias relative to the decision threshold affect the performance of SAT+rwEC. The relative sample size of rwEC to rPh2 may also impact performance. The dashboard can provide quantitative guidance for Ph3-GO if net bias can be estimated by independent studies. Further work to quantify net bias and refine Ph3-GO criteria can help reduce the currently high False-GO rates while increasing opportunities for patients to receive experimental therapies through the SAT+rwEC design. Ph3-GO probability for rPh2 vs. SAT+rwEC with threshold=15%, baseline ORR=20% (select scenarios).[Table: see text]