Estimate Treatment Effects Research Articles

The Evolution of Selection Bias in the Recent Epidemiologic Literature-A Selective Overview.

Selection bias has long been central in methodological discussions across epidemiology and other fields. In epidemiology, the concept of selection bias has been continually evolving over time. In this issue of the Journal, Mathur and Shpitser (Am J Epidemiol. XXXX;XXX(XX):XXXX-XXXX) present simple graphical rules for using a Single World Intervention Graph (SWIG) to assess the presence of selection bias when estimating treatment effects in both the general population and a selected sample. Notably, the authors examine the setting in which the treatment affects selection, an issue not well-addressed in the existing literature on selection bias. To place the work by Mathur and Shpitser in context, we review the evolution of the concept of selection bias in epidemiology, with a primary focus on the developments in the last 20-30 years since the introduction of causal directed acyclic graphs (DAGs) to epidemiologic research.

Two Decades of CABG in the UK: A Propensity Matched Analysis of Outcomes by Conduit Choice.

Background/Objectives: Grafting of LIMA to LAD has long been considered the gold-standard conduit choice for patients undergoing CABG. Despite this, the LSV remains the most used conduit by volume and some patients may not receive even a single arterial conduit. However, the outcomes in this group are not frequently explored. This study, therefore, compares in-hospital outcomes of patients who underwent CABG without any arterial conduits to those who received at least one arterial conduit. Methods: Retrospective propensity-matched database analysis of consecutive patients undergoing CABG in the UK between 1996 and 2019 using data from the National Adult Cardiac Surgery Audit. Results: 335,144 patients underwent CABG, with 6% receiving venous conduits only; matched outcomes are reported for 39,812 patients. In both unmatched and matched groups, we found a significant increase in mortality with the use of veins only (matched mortality 5.3% vs. 3.8%, p < 0.001) with estimated treatment effect for mortality OR 1.43, p < 0.001 (95% CI: 1.31-1.57). We also identified greater rates of post-operative dialysis, IABP insertion, and length of hospital stay in this group. Conclusions: We identified a significant increase in in-hospital mortality with the use of veins only compared to using at least one arterial graft to the LAD. While a single arterial graft should be prioritised wherever possible, venous revascularisation retains a critical role for specific patients. We must, therefore, continue to conduct research addressing the mechanisms underlying and propagating vein graft disease in order better to optimise outcomes for this niche patient group after CABG.

Statistical considerations when estimating time-saving treatment effects in Alzheimer's disease clinical trials.

Estimating treatment effects as time savings in disease progression may be more easily interpretable than assessing the absolute difference or a percentage reduction. In this study, we investigate the statistical considerations of the existing method for estimating time savings and propose alternative complementary methods. We propose five alternative methods to estimate the time savings from different perspectives. These methods are applied to simulated clinical trial data that mimic or modify the Clinical Dementia Rating Sum of Boxes progression trajectories observed in the Clarity AD lecanemab trial. Our study demonstrates that the proposed methods can generate more precise estimates by considering two crucial factors: (1) the absolute difference between treatment arms, and (2) the observed progression rate in the treatment arm. Quantifying treatment effects as time savings in disease progression offers distinct advantages. To provide comprehensive estimations, it is important to use various methods. We explore the statistical considerations of the current method for estimating time savings. We proposed alternative methods that provide time savings estimations based on the observed absolute differences. By using various methods, a more comprehensive estimation of time savings can be achieved.

Minimizing confounding in comparative observational studies with time-to-event outcomes: An extensive comparison of covariate balancing methods using Monte Carlo simulation.

Observational studies are frequently used in clinical research to estimate the effects of treatments or exposures on outcomes. To reduce the effects of confounding when estimating treatment effects, covariate balancing methods are frequently implemented. This study evaluated, using extensive Monte Carlo simulation, several methods of covariate balancing, and two methods for propensity score estimation, for estimating the average treatment effect on the treated using a hazard ratio from a Cox proportional hazards model. With respect to minimizing bias and maximizing accuracy (as measured by the mean square error) of the treatment effect, the average treatment effect on the treated weighting, fine stratification, and optimal full matching with a conventional logistic regression model for the propensity score performed best across all simulated conditions. Other methods performed well in specific circumstances, such as pair matching when sample sizes were large (n = 5000) and the proportion treated was < 0.25. Statistical power was generally higher for weighting methods than matching methods, and Type I error rates were at or below the nominal level for balancing methods with unbiased treatment effect estimates. There was also a decreasing effective sample size with an increasing number of strata, therefore for stratification-based weighting methods, it may be important to consider fewer strata. Generally, we recommend methods that performed well in our simulations, although the identification of methods that performed well is necessarily limited by the specific features of our simulation. The methods are illustrated using a real-world example comparing beta blockers and angiotensin-converting enzyme inhibitors among hypertensive patients at risk for incident stroke.

Optimal designs using generalized estimating equations in cluster randomized crossover and stepped wedge trials.

Cluster randomized crossover and stepped wedge cluster randomized trials are two types of longitudinal cluster randomized trials that leverage both the within- and between-cluster comparisons to estimate the treatment effect and are increasingly used in healthcare delivery and implementation science research. While the variance expressions of estimated treatment effect have been previously developed from the method of generalized estimating equations for analyzing cluster randomized crossover trials and stepped wedge cluster randomized trials, little guidance has been provided for optimal designs to ensure maximum efficiency. Here, an optimal design refers to the combination of optimal cluster-period size and optimal number of clusters that provide the smallest variance of the treatment effect estimator or maximum efficiency under a fixed total budget. In this work, we develop optimal designs for multiple-period cluster randomized crossover trials and stepped wedge cluster randomized trials with continuous outcomes, including both closed-cohort and repeated cross-sectional sampling schemes. Local optimal design algorithms are proposed when the correlation parameters in the working correlation structure are known. MaxiMin optimal design algorithms are proposed when the exact values are unavailable, but investigators may specify a range of correlation values. The closed-form formulae of local optimal design and MaxiMin optimal design are derived for multiple-period cluster randomized crossover trials, where the cluster-period size and number of clusters are decimal. The decimal estimates from closed-form formulae can then be used to investigate the performances of integer estimates from local optimal design and MaxiMin optimal design algorithms. One unique contribution from this work, compared to the previous optimal design research, is that we adopt constrained optimization techniques to obtain integer estimates under the MaxiMin optimal design. To assist practical implementation, we also develop four SAS macros to find local optimal designs and MaxiMin optimal designs.

Perbandingan Propensity Score Stratification dan Propensity Score Matching dengan Pendekatan Multivariate Adaptive Regression Spline

Research on complications of Diabetes Mellitus (DM) is multifactorial, where the risk factors causing DM complications are interrelated, leading to confounding bias, which results in inaccurate research findings. Confounding bias can be reduced using the propensity score method. This study aims to compare the performance of the Propensity Score Stratification (PSS) and Propensity Score Matching (PSM) methods with the Multivariate Adaptive Regression Spline (MARS) approach in estimating treatment effects on DM complication cases. The data used is the medical records of type-2 DM patients at Hospital X. The results showed that the PSS method with the MARS approach is not suitable for small data sets, as it can lead to treatment or control groups lacking members, making it impossible to calculate the p-value in balance testing or the Percent Bias Reduction (PBR). The estimated Average Treatment Effect (ATE) using the PSS method was 0.487 with a PBR of 35.1%, whereas the estimated Average Treatment for Treated (ATT) using the PSM method was 0.531 with a PBR of 99.46%. These PBR values indicate that the best method for estimating treatment effects and the one that can reduce the most bias in this case is the PSM method with MARS. The analysis also showed that serum uric acid levels significantly affect the peripheral diabetic neuropathy (PDN) status of DM patients.

Invited Commentary: Influence of Incomplete Death Information on Cumulative Risk Estimates.

Censoring at death is the only feasible option if death is not recorded and individuals who died simply no longer contribute visits, such as in the setting of Barberio et al (Am J Epidemiol. 2024) before they acquired access to mortality information. Censoring at death is known to lead to biased estimates of the probability of the event of interest before time $t$. Barberio et al show through simulations that this bias increases with increasing mortality. However, when analyzing claims data it is often important to not exclude individuals with shorter life expectancies: an important strength of observational studies is that they allow estimating treatment effects in more varied populations than typically included in randomized clinical trials. We derive an analytic expression for the bias, and provide two upper bounds for the bias. The bounds inform the usefulness of obtaining mortality information. If the probability of death before the event is known to be small, wider confidence intervals can be created using the first bound on the bias; we provide an algorithm. If the bias is large, obtaining mortality information is important. Barberio et al show that obtaining mortality information can be essential in practice.

Challenges of estimating treatment effects after a positive interim analysis

BackgroundThis research investigates why a beneficial treatment effect reported at the first interim analysis (IA) may diminish at a subsequent analysis (SA). We examined three challenges in interpreting treatment effects from randomized clinical trials (RCTs) after the first positive IA: overestimation bias; non-proportional hazards; and heterogeneity in recruitment. We investigate how a penalized estimation method can address overestimation bias, and discuss additional factors to consider when interpreting positive IA results. MethodsWe identified oncology RCTs reporting positive results at the initial IA and a SA for event-free (EFS) and overall survival (OS). We modeled: (1) the hazard ratio at IA (HRIA) versus its timing as measured by the information fraction (IF; i.e., events at IA versus total events sought); and (2), the ratio of HRIA to HRSA (rHR) versus the IF. This was repeated for HRIA adjusted for overestimation bias. Examples of the other two challenges were sought. ResultsAmongst 71 RCTs, HRIA were positively associated with the IF (slope: EFS 0.83, 95 % CI 0.44–1.22; OS 0.25, 95 % CI 0.10–0.41). HRIA tended to exaggerate HRSA, and more so the lower the IF (slope rHR versus IF: EFS 0.10, 95 % CI − 0.22 to 0.42; OS 0.26, 95 % CI 0.07–0.46). Adjusted HRIA did not exaggerate HRSA (slope rHR versus IF: EFS − 0.14, 95 % CI − 0.67 to 0.39; OS 0.02, 95 % CI − 0.26 to 0.30). Examples of two other challenges are shown. ConclusionOverestimation bias, non-proportional hazards, and heterogeneity in recruitment and other important treatments should be considered when communicating estimates of treatment effects from positive IAs.

A simple nonparametric least-squares-based causal inference for heterogeneous treatment effects

Estimating treatment effects is a common practice in making causal inferences. However, it is a challenging task for observational studies because the underlying models for outcome and treatment assignment are unknown. The concept of potential outcomes has been widely adopted in the literature on causal inferences. Building on potential outcomes, we propose a simple nonparametric least-squares spline-based causal inference method to estimate heterogeneous treatment effects in this manuscript. We use empirical process theory to study its asymptotic properties and conduct simulation studies to evaluate its operational characteristics. Based on the estimated heterogeneous treatment effects, we further estimate the average treatment effect and show the asymptotic normality of the estimator. Finally, we apply the proposed method to assess the biological anti-rheumatic treatment effect on children with newly onset juvenile idiopathic arthritis disease using electronic health records from a longitudinal study at Cincinnati Children's Hospital Medical Center.

Heart failure events in randomized controlled trials for adults receiving maintenance dialysis: a meta-epidemiologic study.

Heart failure is characterized as cardiac dysfunction resulting in elevated cardiac filling pressures with symptoms and signs of congestion. Distinguishing heart failure from other causes of similar presentations in patients with kidney failure is challenging but necessary, and is needed in randomized controlled trials (RCTs) to accurately estimate treatment effects. The objective of this study was to review heart failure events, their diagnostic criteria and adjudication in RCTs of patients with kidney failure treated with dialysis. We hypothesized that heart failure events, diagnostic criteria and adjudication were infrequently reported in RCTs in dialysis. We conducted a meta-epidemiologic systematic review of RCTs from high impact medical, nephrology and cardiology journals from 2000 to 2020. RCTs were eligible if they enrolled adults receiving maintenance dialysis for kidney failure and evaluated any intervention. Results. Of 561 RCTs in patients receiving dialysis, 36 (6.4%) reported heart failure events as primary (10, 27.8%) or secondary (31, 86.1%) outcomes. 10 of the 36 (27.8%) RCTs provided heart failure event diagnostic criteria and 5 of these 10 (50%) adjudicated heart failure events. These 10 RCTs included event diagnostic criteria for heart failure or heart failure hospitalizations, and their criteria included dyspnea (5/10), edema (2/10), rales/crackles (4/10), chest x-ray pulmonary edema or vascular redistribution (4/10), treatment in an acute setting (6/10) and ultrafiltration or dialysis (4/10). No study explicitly distinguished heart failure from volume overload secondary to non-adherence or underdialysis. Overall, we found that heart failure events are infrequently reported in RCTs in dialysis and are heterogeneously defined. Further research is required to develop standardized diagnostic criteria that are practical and meaningful to patients and clinicians.

Revisiting the connection between state Medicaid expansions and adult mortality

AbstractThis paper examines the impact of Medicaid expansions to parents and childless adults on adult mortality. Specifically, we evaluate the long‐run effects of eight state Medicaid expansions from 1994 through 2005 on all‐cause, healthcare‐amenable, non‐healthcare‐amenable, and HIV‐related mortality rates using state‐level data. We utilize the synthetic control method to estimate effects for each treated state separately and the generalized synthetic control method to estimate average effects across all treated states. Using a 5% significance level, we find no evidence that Medicaid expansions affect any of the outcomes in any of the treated states or all of them combined. Moreover, there is no clear pattern in the signs of the estimated treatment effects. These findings imply that evidence that pre‐ACA Medicaid expansions to adults saved lives is not as clear as previously suggested.

Propensity Weighted federated learning for treatment effect estimation in distributed imbalanced environments

Estimating treatment effects from observational data in medicine using causal inference is a very relevant task due to the abundance of observational data and the ethical and cost implications of conducting randomized experiments or experimental interventions. However, how could we estimate the effect of a treatment in a hospital that has very restricted access to treatment? In this paper, we want to address the problem of distributed causal inference, where hospitals not only have different distributions of patients, but also different treatment assignment criteria. Furthermore, it is necessary to take into account that due to privacy restrictions, personal patient data cannot be shared between hospitals. To address this problem, we propose an adaptation of the federated learning algorithm FederatedAveraging to one of the most advanced models for the prediction of treatment effects based on neural networks, TEDVAE. Our algorithm adaptation takes into account the shift in the treatment distribution between hospitals and is therefore called Propensity WeightedFederatedAveraging (PW FedAvg). As the distributions of the assignment of treatments become more unbalanced between the nodes, the estimation of causal effects becomes more challenging. The experiments show that PW FedAvg manages to reduce errors in the estimation of individual causal effects when imbalances are large, compared to VanillaFedAvg and other federated learning-based causal inference algorithms based on the application of federated learning to linear parametric models, Gaussian Processes and Random Fourier Features.

Effects of basic carbohydrate counting versus standard dietary care for glycaemic control in type 2 diabetes (The BCC Study): a randomised, controlled trial

BackgroundClinical guidelines recommend basic carbohydrate counting (BCC), or similar methods to improve carbohydrate estimation skills and to strive for higher consistency in carbohydrate intake potentially improving glycaemic control. However, evidence for this approach in type 2 diabetes (T2D) is limited.ObjectiveTo examine the efficacy of a structured education program in BCC as add-on to standard dietary care on glycaemic control in individuals with T2D.MethodsThe BCC Study was a randomized, controlled, open-label, parallel-group trial. Individuals with T2D aged 18-75 years with glycated haemoglobin A1c (HbA1c) 53–97 mmol/mol (7.0–11.0%) were randomly assigned (1:1) to BCC or standard dietary care. The primary outcomes were differences in changes in HbA1c or glycaemic variability (calculated as mean amplitude of glycaemic excursions [MAGE]) between groups after six months of intervention.ResultsBetween September 2018 and July 2021, 48 participants were randomly assigned, 23 to BCC and 25 to standard dietary care. Seven participants did not receive the allocated intervention. From a baseline-adjusted mean of 65 mmol/mol (95% CI 62-68 [8.1%, 7.8-8.4]), HbA1c changed by −5 mmol/mol (−8 to −1 [−0.5%, −0.7 to −0.1]) in BCC and -3 mmol/mol (−7 to 1 [−0.3%, −0.6 to 0.1]) in standard care with an estimated treatment effect of −2 mmol/mol (−7 to 4 [−0.2%, −0.6 to 0.4]); p = 0.554. From a baseline-adjusted mean of 4.2 mmol/l (3.7 to 4.8), MAGE changed by −16% (−33 to 5) in BCC and by −3% (−21 to 20) in standard care with an estimated treatment effect of −14% (−36 to 16); p = 0.319. Only median carbohydrate estimation error in favour of BCC (estimated treatment difference −55% (−70 to −32); p < 0.001) remained significant after multiple testing adjustment.ConclusionsNo glycaemic effects were found but incorporating BCC as a supplementary component to standard dietary care led to improved skills in estimating carbohydrate intake among individuals with T2D.

Novel Autologous Dendritic Cell Therapy AVT001 for Type 1 Diabetes

BackgroundCD8+ T regulatory (Treg) cells that recognize the nonclassical class 1b molecule Qa-1/human leukocyte antigen E (Q/E CD8+ Treg cells) are important in maintaining self-tolerance. We sought to investigate the role that these T cells play in type 1 diabetes (T1D) pathogenesis and whether an intervention targeting this mechanism may delay T1D progression.MethodsWe conducted a phase 1/2, randomized, double-blind, placebo-controlled trial of the autologous dendritic cell therapy AVT001 that included participants at least 16 years of age, within 1 year of T1D diagnosis, and with ex vivo evidence of a defect in Q/E CD8+ Treg function. Patients were randomly assigned in a 2:1 ratio to AVT001 or placebo, which was administered in three monthly intravenous infusions. The primary end point was safety; efficacy end points included changes from baseline in C-peptide area under the curve (AUC) during a 4-hour mixed meal, hemoglobin A1c (HbA1c), and insulin dose.ResultsSixteen patients received AVT001, and nine received placebo. Similar rates and severity of adverse events were observed in both groups. None of the patients in the AVT001 group had serious adverse events through visit day 360. Compared with placebo, treatment with ATV001 was associated with less decline from baseline log-transformed C-peptide AUC (nmol/l), with the treatment effect between AVT001 and placebo at day 150 of 0.09 (95% confidence interval [CI], 0.03 to 0.15) and at day 360 of 0.10 (95% CI, 0.04 to 0.15). No clear differences in change in HbA1c and insulin dose from baseline were observed between groups. Estimated treatment effects of AVT001 versus placebo at day 360 were −0.17% (95% CI, −0.60 to 0.26%) for HbA1c and −0.06 U/kg/day (95% CI, −0.14 to 0.02) for daily insulin dose.ConclusionsIn this phase 1/2 trial, AVT001 did not result in dose-limiting adverse events. Potential signals of efficacy observed here warrant further evaluation in a fully powered trial. (Funded by Avotres Inc. and the Division of Diabetes, Endocrinology, and Metabolic Diseases; ClinicalTrials.gov number, NCT03895996.)

Large pre-trained models for treatment effect estimation: Are we there yet?

Deep learning for causal inference is a promising technique that leverages deep neural networks to infer counterfactuals and estimate treatment effects. Liu etal. proposed CURE (causal treatment effect estimation), a new pre-training and fine-tuning framework for treatment effects estimation using large-scale patient data.

Multiple imputation for propensity score analysis with covariates missing at random: some clarity on "within" and "across" methods.

In epidemiology and the social sciences, propensity score methods are popular for estimating treatment effects using observational data, and multiple imputation is popular for handling covariate missingness. However, how to appropriately use multiple imputation for propensity score analysis is not completely clear. This paper aims to bring clarity on the consistency (or lack thereof) of methods that have been proposed, focusing on the "within" approach (where the effect is estimated separately in each imputed dataset and then the multiple estimates are combined) and the "across" approach (where typically propensity scores are averaged across imputed datasets before being used for effect estimation). We show that the within method is valid and can be used with any causal effect estimator that is consistent in the full-data setting. Existing across methods are inconsistent, but a different across method that averages the inverse probability weights across imputed datasets is consistent for propensity score weighting. We also comment on methods that rely on imputing a function of the missing covariate rather than the covariate itself, including imputation of the propensity score and of the probability weight. Based on consistency results and practical flexibility, we recommend generally using the standard within method. Throughout, we provide intuition to make the results meaningful to the broad audience of applied researchers.

Blood-based markers of differential efficacy of bipolar androgen therapy and enzalutamide in the randomized TRANSFORMER trial.

LBA5014 Background: Bipolar androgen therapy (BAT) for metastatic castration-resistant prostate cancer (mCRPC) is administered by alternating between supraphysiologic and near-castrate serum testosterone levels through intramuscular administration of testosterone cypionate 400mg given every 28 days together with ongoing androgen suppression. BAT is effective in a subset of patients, but evidence for predictive treatment selection is lacking. The goal of the study is to determine whether blood-based markers can be identified in the TRANSFORMER study, a randomized trial of abiraterone-pretreated mCRPC patients assigned to BAT or enzalutamide (Enza). Methods: We conducted whole genome and whole exome sequencing of circulating tumor DNA samples collected from the TRANSFORMER study. In this post-hoc biomarker study of a randomized trial, we sought to identify markers that predict preferential benefit from BAT or Enza in mCRPC patients progressing on abiraterone. We compared clinical or radiographic progression-free survival (PFS) and overall survival (OS) between BAT and Enza arms in subgroups defined by biomarker status and estimated treatment effects via the Cox regression model, stratified by duration of prior abiraterone treatment. To determine whether a molecular event is a predictive biomarker, we tested the interaction term of the dichotomized marker status by treatment arms. Statistical tests were two-sided, and p values ≤ 0.05 were deemed to indicate statistical significance. Results: We focused on somatic alterations implicated in androgen receptor (AR) signaling that can be detected at a relatively high frequency in blood samples. Whole genome sequencing and whole exome sequencing of cell-free DNA from 62 patients revealed tumor-specific AR pathway alterations, including AR point mutations and amplifications (33/62, 53.2%). In men with positive AR alterations detected in blood, BAT was more efficacious than Enza (median PFS 4.2 months vs. 2.9 months; hazard ratio [HR] 0.59[95% CI 0.25-1.37], P=0.22), while Enza was superior to BAT in those without AR alterations (median PFS 8.4 months vs. 3 months; HR 3.62[95%CI 1.44-9.1], P=0.006). We detected a significant interaction between AR alteration status and treatment types using PFS as the endpoint ( Pinteraction=0.002). The differential benefit is upheld with OS as the endpoint ( Pinteraction &lt;0.001). In men with positive AR alterations, OS was longer with BAT compared with Enza (median OS 29.6 months vs. 24.1 months; HR 0.41[95% CI 0.16-1.03], P=0.058). In contrast, in men without AR alterations, OS was worse with BAT compared with Enza (median OS 19.3 months vs. NR; HR 4.38[95% CI 1.21-15.89], P=0.025). Conclusions: Metastatic CRPC patients progressing on abiraterone with AR alterations detected in blood may benefit preferentially from BAT. Routine liquid biopsy testing may enable further adoption of BAT.

An Efficient Way to Find Optimal Crossover Designs Using CVX for Precision Medicine

Crossover designs play an increasingly important role in precision medicine. We show the search of an optimal crossover design can be formulated as a convex optimization problem and convex optimization tools, such as CVX, can be directly used to search for an optimal crossover design. We first demonstrate how to transform crossover design problems into convex optimization problems and show CVX can effortlessly find optimal crossover designs that coincide with a few theoretical crossover optimal designs in the literature. The proposed approach is especially useful when it becomes problematic to construct optimal designs analytically for complicated models. We then apply CVX to find crossover designs for models with auto-correlated error structures or when the information matrices may be singular and analytical answers are unavailable. We also construct N-of-1 trials frequently used in precision medicine to estimate treatment effects on the individuals or to estimate average treatment effects, including finding dual-objective optimal crossover designs.

Early vs Late Anticoagulation in Minor, Moderate, and Major Ischemic Stroke With Atrial Fibrillation

Whether infarct size modifies the treatment effect of early vs late direct oral anticoagulant (DOAC) initiation in people with ischemic stroke and atrial fibrillation is unknown. To assess whether infarct size modifies the safety and efficacy of early vs late DOAC initiation. Post hoc analysis of participants from the multinational (>100 sites in 15 countries) randomized clinical Early Versus Later Anticoagulation for Stroke With Atrial Fibrillation (ELAN) trial who had (1) acute ischemic stroke, (2) atrial fibrillation, and (3) brain imaging available before randomization. The ELAN trial was conducted between October 2017 and December 2022. Data were analyzed from October to December 2023 for this post hoc analysis. Early vs late DOAC initiation after ischemic stroke. Early DOAC initiation was within 48 hours for minor or moderate stroke or on days 6 to 7 for major stroke; late DOAC initiation was on days 3 to 4 for minor stroke, days 6 to 7 for moderate stroke, and days 12 to 14 for major stroke. The primary outcome was a composite of recurrent ischemic stroke, symptomatic intracranial hemorrhage, extracranial bleeding, systemic embolism, or vascular death within 30 days. The outcome was assessed according to infarct size (minor, moderate, or major) using odds ratios and risk differences between treatment arms. Interrater reliability for infarct size between the core laboratory and local raters was assessed, and whether this modified the estimated treatment effects was also examined. A total of 1962 of the original 2013 participants (909 [46.3%] female; median [IQR] age, 77 [70-84] years) were included. The primary outcome occurred in 10 of 371 participants (2.7%) with early DOAC initiation vs 11 of 364 (3.0%) with late DOAC initiation among those with minor stroke (odds ratio [OR], 0.89; 95% CI, 0.38-2.10); in 11 of 388 (2.8%) with early DOAC initiation vs 14 of 392 (3.6%) with late DOAC initiation among those with moderate stroke (OR, 0.80; 95% CI, 0.35-1.74); and in 8 of 219 (3.7%) with early DOAC initiation vs 16 of 228 (7.0%) with late DOAC initiation among those with major stroke (OR, 0.52; 95% CI, 0.21-1.18). The 95% CI for the estimated risk difference of the primary outcome in early anticoagulation was -2.78% to 2.12% for minor stroke, -3.23% to 1.76% for moderate stroke, and -7.49% to 0.81% for major stroke. There was no significant treatment interaction for the primary outcome. For infarct size, interrater reliability was moderate (κ = 0.675; 95% CI, 0.647-0.702) for local vs core laboratory raters and strong (κ = 0.875; 95% CI, 0.855-0.894) between core laboratory raters. The treatment effect of early DOAC initiation did not differ in people with minor, moderate, or major stroke assessed by brain imaging. Early treatment was not associated with a higher rate of adverse events, especially symptomatic intracranial hemorrhage, for any infarct size, including major stroke. ClinicalTrials.gov Identifier: NCT03148457.

Assessing external validity in practice

We review, from a practical standpoint, the evolving literature on assessing external validity (EV) of estimated treatment effects. We review existing EV measures, and focus on methods that permit multiple datasets (Hotz et al., 2005). We outline criteria for practical usage, evaluate the existing approaches, and identify a gap in potential methods. Our practical considerations motivate a novel method utilizing the Group Lasso (Yuan and Lin, 2006) to estimate a tractable regression-based model of the conditional average treatment effect (CATE). This approach can perform better when settings have differing covariate distributions and allows for easily extrapolating the average treatment effect to new settings. We apply these measures to a set of identical field experiments upgrading slum dwellings in three different countries (Galiani et al., 2017).