Pseudo-observations and super learner for the estimation of the restricted mean survival time.

Background:Restricted mean survival time is a measure of average survival time up to a specified time point. There has been an increased interest in using restricted mean survival time to compare treatment arms in randomized clinical trials because such comparisons do not rely on proportional hazards or other assumptions about the nature of the relationship between survival curves.Methods:This article addresses the question of whether covariate adjustment in randomized clinical trials that compare restricted mean survival times improves precision of the estimated treatment effect (difference in restricted mean survival times between treatment arms). Although precision generally increases in linear models when prognostic covariates are added, this is not necessarily the case in non-linear models. For example, in logistic and Cox regression, the standard error of the estimated treatment effect does not decrease when prognostic covariates are added, although the situation is complicated in those settings because the estimand changes as well. Because estimation of restricted mean survival time in the manner described in this article is also based on a model that is non-linear in the covariates, we investigate whether the comparison of restricted mean survival times with adjustment for covariates leads to a reduction in the standard error of the estimated treatment effect relative to the unadjusted estimator or whether covariate adjustment provides no improvement in precision. Chen and Tsiatis suggest that precision will increase if covariates are chosen judiciously. We present results of simulation studies that compare unadjusted versus adjusted comparisons of restricted mean survival time between treatment arms in randomized clinical trials.Results:We find that for comparison of restricted means in a randomized clinical trial, adjusting for covariates that are associated with survival increases precision and therefore statistical power, relative to the unadjusted estimator. Omitting important covariates results in less precision but estimates remain unbiased.Conclusion:When comparing restricted means in a randomized clinical trial, adjusting for prognostic covariates can improve precision and increase power.

The restricted mean survival time (RMST) is an appealing measurement in clinical or epidemiological studies with censored survival outcome and receives a lot of attention in the past decades. It provides a useful alternative to the Cox model for evaluating the covariate effect on survival time. The covariate effect on RMST usually varies with the restriction time. However, existing methods cannot address this problem properly. In this article, we propose a semiparametric framework that directly models RMST as a function of the restriction time. Our proposed model adopts a widely-used proportional form, enabling the estimation of RMST predictions across an interval using a unified model. Furthermore, the covariate effect for multiple restriction time points can be derived simultaneously. We develop estimators based on estimating equations theories and establish the asymptotic properties of the proposed estimators. The finite sample properties of the estimators are evaluated through extensive simulation studies. We further illustrate the application of our proposed method through the analysis of two real data examples. Supplementary Material are available online.

Azacitidine Treatment in MDS: A Systematic Literature Review and Meta-Analysis Comparing the Efficacy of Real World Data with Randomized Controlled Trials

When trying to learn a model for the prediction of an outcome given a set of covariates, a statistician has many estimation procedures in their toolbox. A few examples of these candidate learners are: least squares, least angle regression, random forests, and spline regression. Previous articles (van der Laan and Dudoit (2003); van der Laan et al. (2006); Sinisi et al. (2007)) theoretically validated the use of cross validation to select an optimal learner among many candidate learners. Motivated by this use of cross validation, we propose a new prediction method for creating a weighted combination of many candidate learners to build the super learner. This article proposes a fast algorithm for constructing a super learner in prediction which uses V-fold cross-validation to select weights to combine an initial set of candidate learners. In addition, this paper contains a practical demonstration of the adaptivity of this so called super learner to various true data generating distributions. This approach for construction of a super learner generalizes to any parameter which can be defined as a minimizer of a loss function.

The restricted mean survival time (RMST) is a clinically meaningful summary measure in studies with survival outcomes. Statistical methods have been developed for regression analysis of RMST to investigate impacts of covariates on RMST, which is a useful alternative to the Cox regression analysis. However, existing methods for regression modeling of RMST are not applicable to left-truncated right-censored data that arise frequently in prevalent cohort studies, for which the sampling bias due to left truncation and informative censoring induced by the prevalent sampling scheme must be properly addressed. The pseudo-observation (PO) approach has been used in regression modeling of RMST for right-censored data and competing-risks data. For left-truncated right-censored data, we propose to directly model RMST as a function of baseline covariates based on POs under general censoring mechanisms. We adjust for the potential covariate-dependent censoring or dependent censoring by the inverse probability of censoring weighting method. We establish large sample properties of the proposed estimators and assess their finite sample performances by simulation studies under various scenarios. We apply the proposed methods to a prevalent cohort of women diagnosed with stage IV breast cancer identified from surveillance, epidemiology, and end results-medicare linked database.

8524 Background: Pembrolizumab-based chemo-immunotherapies (Pembro) and nivolumab plus ipilimumab-based immunotherapies with or without 2 cycles of chemotherapies (Nivo+Ipi) have improved survival in patients with advanced NSCLC compared to the conventional chemotherapy. However, biomarkers to support appropriate choice in these immunotherapies remain unclear. Methods: From 2019 to 2023, this multicenter, observational study retrospectively reviewed advanced NSCLC patients who received first-line Pembro or Nivo+Ipi and had evaluable PD-L1 expression status on tumor cells (tumor proportion score [TPS], 22C3) and immune cells (immune cell [IC] score, SP142). Survival curve comparisons between treatments were conducted using restricted mean survival time (RMST) estimation in place of Log-rank test, when the proportional hazard assumption was not met. Additionally, the genomic and expression profiles associated with TPS and IC score were assessed using whole-exome sequencing and RNA sequencing in available NSCLC samples. Results: A total of 198 patients were included (Pembro/Nivo+Ipi: 137/61). In the Pembro cohort, patients with high TPS (≥ 50%) had significantly longer progression-free survival (PFS) than those with low TPS (< 50%) (median PFS [mPFS, months]: 8.1 vs. 7.1, P = 0.02; hazard ratio [HR] = 0.59 [0.38–0.92]), while no significant difference in PFS was observed based on IC score (high vs. low: mPFS 7.4 vs. 6.8, P = 0.11, HR = 0.72 [0.49–1.07]). In the Nivo+Ipi cohort, PFS did not significantly differ by TPS (high vs. low: mPFS 4.0 vs. 4.0, P = 0.26; HR = 0.51 [0.16–1.68]), whereas patients with high IC score (≥ 1) had significantly longer PFS than those with low IC score (= 0) (mPFS: 7.7 vs. 2.8, P = 0.04; HR = 0.53 [0.28–0.98]). A durable PFS benefit of Nivo+Ipi over Pembro was observed only in patients with low TPS/high IC score (mPFS: 12.4 vs. 6.6; Schoenfeld individual test: P < 0.05; RMST Nivo+Ipi /RMST Pembro [2 years] = 1.5, P = 0.049, Table). Sequence analyses revealed that tumors with low TPS/high IC score had significantly higher tumor mutational burden (TMB) than other tumors (median TMB: 18.2 vs. 1.9 [/mb]; P < 0.001) and showed distinct enrichment in antigen presentation and T-cell receptor signaling pathways. Conclusions: Nivolumab plus ipilimumab-based immunotherapies demonstrated superior durable response compared to pembrolizumab-based chemo-immunotherapies in patients with low TPS/high IC score. PD-L1 phenotypes based on TPS and IC score could guide the optimal selection of immunotherapies for advanced NSCLC patients. Efficacy comparison in patients with low TPS (< 50%)/high IC score (≥ 1). Treatments mPFS (months) PFS rate at 2 years RMST at 2 years RMST Nivo+Ipi /RMST Pembro at 2 years P value Pembrolizumab-based chemo-immunotherapies 6.6 6% 8.5 1.5 [1.0–2.3] 0.049 Nivolumab plus ipilimumab-based immunotherapies 12.4 41% 12.9

In oncology studies, the assumption of proportional hazards is often questionable due to factors such as the presence of cured patients, a delayed treatment benefit, and possible treatment switching. The restricted mean survival time (RMST) has emerged as a valuable alternative summary measure to the hazard ratio (HR) in this scenario as it provides a clinically meaningful interpretation of treatment benefit without additional assumptions. As a commonly used primary endpoint, progression-free survival (PFS) is defined as the time from randomization to the first occurrence of death or progression of disease (PD). However, PFS involves dual observation processes where, in practice, the exact death time is typically recorded, but PD is interval-censored. This feature is also present in other commonly used primary endpoints, including event-free survival, disease-free survival, and relapse-free survival. The conventional approach imputes the PD time with the right boundary of the time interval during which the PD occurs. This paper presents alternative estimation and inference approaches to estimate RMST with a mixture of right-censored and interval-censored data. Different approaches are explored by simulation under various plausible scenarios for oncology clinical trials with regard to the assessment frequency, randomness in the actual assessment times, and size of treatment effect. The choice of the restricted time point in RMST is also explored. The simulation results indicate that the RMST estimators that take account of the interval censoring inherent in the data are unbiased and more accurate than the conventional estimators, while the performance for two-group comparisons is comparable. Furthermore, the performance of the proposed estimators is contingent on the scheduled assessment plan and patients' visit window.

Combined Cycle Power Plants (CCPPs) generate electrical power through gas turbines and use the exhaust heat from those turbines to power steam turbines, resulting in 50% more power output compared to traditional simple cycle power plants. Predicting the full-load electrical power output (PE) of a CCPP is crucial for efficient operation and sustainable development. Previous studies have used machine learning models, such as the Bagging and Boosting models to predict PE. In this study, we propose employing Super Learner (SL), an ensemble machine learning algorithm, to enhance the accuracy and robustness of predictions. SL utilizes cross-validation to estimate the performance of diverse machine learning models and generates an optimal weighted average based on their respective predictions. It may provide information on the relative contributions of each base learner to the overall prediction skill. For constructing the SL, we consider six individual and ensemble machine learning models as base learners and assess their performances compared to the SL. The dataset used in this study was collected over six years from an operational CCPP. It contains one output variable and four input variables: ambient temperature, atmospheric pressure, relative humidity, and vacuum. The results show that the Boosting algorithms significantly influence the performance of the SL in comparison to the other base learners. The SL outperforms the six individual and ensemble machine learning models used as base learners. It indicates that the SL improves the generalization performance of predictions by combining the predictions of various machine learning models.

In clinical follow-up studies with a time-to-event end point, the difference in the restricted mean survival time (RMST) is a suitable substitute for the hazard ratio (HR). However, the RMST only measures the survival of patients over a period of time from the baseline and cannot reflect changes in life expectancy over time. Based on the RMST, we study the conditional restricted mean survival time (cRMST) by estimating life expectancy in the future according to the time that patients have survived, reflecting the dynamic survival status of patients during follow-up. In this paper, we introduce the estimation method of cRMST based on pseudo-observations, the statistical inference concerning the difference between two cRMSTs (cRMSTd), and the establishment of the robust dynamic prediction model using the landmark method. Simulation studies are conducted to evaluate the statistical properties of these methods. The results indicate that the estimation of the cRMST is accurate, and the dynamic RMST model has high accuracy in coefficient estimation and good predictive performance. In addition, an example of patients with chronic kidney disease who received renal transplantations is employed to illustrate that the dynamic RMST model can predict patients' expected survival times from any prediction time, considering the time-dependent covariates and time-varying effects of covariates.

ABSTRACTThe optimal learner for prediction modeling varies depending on the underlying data-generating distribution. Super Learner (SL) is a generic ensemble learning algorithm that uses cross-validation to select among a ‘library’ of candidate prediction models. While SL has been widely studied in a number of settings, it has not been thoroughly evaluated in large electronic healthcare databases that are common in pharmacoepidemiology and comparative effectiveness research. In this study, we applied and evaluated the performance of SL in its ability to predict the propensity score (PS), the conditional probability of treatment assignment given baseline covariates, using three electronic healthcare databases. We considered a library of algorithms that consisted of both nonparametric and parametric models. We also proposed a novel strategy for prediction modeling that combines SL with the high-dimensional propensity score (hdPS) variable selection algorithm. Predictive performance was assessed using three metrics: the negative log-likelihood, area under the curve (AUC), and time complexity. Results showed that the best individual algorithm, in terms of predictive performance, varied across datasets. The SL was able to adapt to the given dataset and optimize predictive performance relative to any individual learner. Combining the SL with the hdPS was the most consistent prediction method and may be promising for PS estimation and prediction modeling in electronic healthcare databases.

328 Background: Clinical pathways are decision support tools designed to enhance accessibility and ease of use of treatment recommendations. Prior studies that evaluated cancer clinical pathways largely focused on process measures such as concordance with treatment guidelines, but little evidence is available about the effect of clinical pathways on survival. We thus aimed to evaluate the effect of cancer clinical pathways on overall survival. Methods: We used institutional registry data from the JPS Oncology and Infusion Center, which is a Comprehensive Community Cancer Program. Eligible patients were aged ≥18 years, diagnosed with first primary breast, colorectal, or lung cancer between December 2013 and December 2017 with follow-up through June 2020. We used a natural experiment framework, where intervention was implementation of clinical pathways (December 2015). The pre-intervention period was December 2013 – November 2015 and post-intervention period was January 2016 – December 2017. We used marginal structural models with stabilized inverse probability weights for each cancer type to estimate restricted mean survival time (RMST) differences and 95% confidence limits (CL) comparing overall survival between pre- and post-clinical pathways within a 36-month horizon, where weights were based on a minimal sufficient set of covariates (sociodemographics, tumor characteristics, and comorbidities) to reduce confounding bias. Results: Our study population comprised 327 breast, 254 colorectal, and 431 lung cancer patients. Median age ranged between 55 years (breast) and 60 years (lung). The frequency of racial/ethnic minorities ranged between 45% (lung) and 68% (colorectal). The frequency of uninsured patients ranged between 47% (lung) and 58% (colorectal). Adherence to pathways was consistently over 80%. RMST was largely similar between pre- and post-clinical pathways for breast (RMST difference = -0.10 months, 95% CL: -1.9, 1.7), colorectal (RMST difference = -0.48 months, 95% CL: -3.4, 2.5) and lung cancer (RMST difference = 1.1 month, 95% CL: -1.5, 3.7). Conclusions: Our results suggest that implementation of clinical pathways has minimal effect on overall survival within 36 months despite high adherence in a safety-net population, but the estimates are compatible with up to 3.4 month decrease or 3.7 month increase depending on cancer type. Longer follow-up, particularly for breast and colorectal cancer, may provide further insight. Our findings may be useful for informing deliberations about implementing clinical pathways and setting expectations about the effect of clinical pathways on overall survival.

Consistency of the propensity score estimators rely on correct specification of the propensity score model. The propensity score is frequently estimated using a main effect logistic regression. It has recently been shown that the use of ensemble machine learning algorithms, such as the Super Learner, could improve covariate balance and reduce bias in a meaningful manner in the case of serious model misspecification for treatment assignment. However, the loss functions normally used by the Super Learner may not be appropriate for propensity score estimation since the goal in this problem is not to optimize propensity score prediction but rather to achieve the best possible balance in the covariate distribution between treatment groups. In a simulation study, we evaluated the benefit of a modification of the Super Learner by propensity score estimation geared toward achieving covariate balance between the treated and untreated after matching on the propensity score. Our simulation study included six different scenarios characterized by various degrees of deviation from the usual main term logistic model for the true propensity score and outcome as well as the presence (or not) of instrumental variables. Our results suggest that the use of this adapted Super Learner to estimate the propensity score can further improve the robustness of propensity score matching estimators.

Likelihood of alcohol dependence (AD) is increased among people who transition to greater levels of alcohol involvement at a younger age. Indicated interventions delivered early may be effective in reducing risk, but could be costly. One way to increase cost-effectiveness would be to develop a prediction model that targeted interventions to the subset of youth with early alcohol use who are at highest risk of subsequent AD. A prediction model was developed for DSM-IV AD onset by age 25 years using an ensemble machine-learning algorithm known as 'Super Learner'. Shapley additive explanations (SHAP) assessed variable importance. Respondents reporting early onset of regular alcohol use (i.e. by 17 years of age) who were aged 25 years or older at interview from 14 representative community surveys conducted in 13 countries as part of WHO's World Mental Health Surveys. The primary outcome to be predicted was onset of life-time DSM-IV AD by age 25 as measured using the Composite International Diagnostic Interview, a fully structured diagnostic interview. AD prevalence by age 25 was 5.1% among the 10 687 individuals who reported drinking alcohol regularly by age 17. The prediction model achieved an external area under the curve [0.78; 95% confidence interval (CI) = 0.74-0.81] higher than any individual candidate risk model (0.73-0.77) and an area under the precision-recall curve of 0.22. Overall calibration was good [integrated calibration index (ICI) = 1.05%]; however, miscalibration was observed at the extreme ends of the distribution of predicted probabilities. Interventions provided to the 20% of people with highest risk would identify 49% of AD cases and require treating four people without AD to reach one with AD. Important predictors of increased risk included younger onset of alcohol use, males, higher cohort alcohol use and more mental disorders. A risk algorithm can be created using data collected at the onset of regular alcohol use to target youth at highest risk of alcohol dependence by early adulthood. Important considerations remain for advancing the development and practical implementation of such models.

Dynamic prediction and analysis based on restricted mean survival time in survival analysis with nonproportional hazards

Association of tumour burden with the efficacy of programmed cell death-1/programmed cell death ligand-1 inhibitors for treatment-naïve advanced non-small-cell lung cancer