Heterogeneous Treatment Effects Research Articles

Heterogeneity of Treatment Effects of Laser Epilation on Pilonidal Disease Recurrence: A Randomized Clinical Trial.

To investigate the heterogeneity of treatment effects (HTE) of laser epilation in preventing pilonidal disease recurrence through analysis of prespecified clinical factors. Pilonidal disease is a common, painful disease affecting 1% of the population aged 15 to 30 years with postoperative recurrence rates as high as 30% to 40%. Single-institution randomized controlled trial from September 2017 to September 2022 with 1-year follow-up, including patients aged 11 to 21 years with pilonidal disease undergoing gluteal cleft laser epilation and standard care (improved hygiene and mechanical or chemical depilation) or standard care alone. In total, 302 patients were enrolled with 151 randomized to each intervention. 1-year follow-up was available for 96 patients in the laser group and 134 in the standard care group. There were no significant differences in treatment effects based on sex, body mass index, previous disease, prior surgical excision, or annual household income (all P > 0.05). HTE was identified by race and ethnicity (P = 0.005) and health insurance type (P = 0.001). Recurrence among non-Hispanic white patients was 4% (3/75) with laser treatment and 31.6% (31/98) with standard care versus 38.9% (7/18) with laser treatment and 38.2% (13/34) with standard care among all other racial/ethnic groups. Recurrence rates among privately insured patients were 4.0% (3/75) with laser treatment and 33.3% (29/87) with standard care versus 36.8% (7/19) with laser treatment and 29.7% (11/37) with standard care in patients with public insurance. The effectiveness of laser epilation to reduce pilonidal disease recurrence rates may vary based on race and ethnicity and insurance type. Additional studies are warranted to investigate this potential HTE.

A 2-stage model of heterogenous treatment effects for brain atrophy in multiple sclerosis utilizing the MS PATHS research network

BackgroundTwo-stage models of heterogenous treatment effects (HTE) may advance personalized medicine in multiple sclerosis (MS). Brain atrophy is a relatively objective outcome measure that has strong relationships to MS prognosis and treatment effects and is enabled by standardized MRI. ObjectivesTo predict brain atrophy outcomes for patients initiating disease-modifying therapies (DMT) with different efficacies, considering the patients’ baseline brain atrophy risk measured via brain parenchymal fraction (BPF). MethodsAnalyses included patients enrolled in the Multiple Sclerosis Partners Advancing Technology and Health Solutions (MS PATHS) network who started DMT and had complete baseline data and ≥ 6-month brain MRI follow-up. All brain MRIs were acquired using standardized acquisition sequences on Siemens 3T scanners. BPF change risk was derived by linear mixed effects models using baseline covariates. Model performance was assessed by predicted versus actual BPF change R2. Propensity score (PS) weighting was used to balance covariates between groups defined by DMT efficacy (high: natalizumab, alemtuzumab, ocrelizumab, and rituximab; moderate: dimethyl fumarate, fingolimod, and cladribine; low: teriflunomide, interferons, and glatiramer acetate). HTE models predicting 1 year change in BPF were built using a weighted linear mixed effects model with low-efficacy DMT as the reference. ResultsAnalyses included 581 high-, 183 moderate-, and 106 low-efficacy DMT-treated patients. The mean and median number of brain MRI observations per treatment period were 2.9 and 3.0, respectively. Risk model performance R2=0.55. After PS weighting, covariate standardized mean differences were <10 %, indicating excellent balance across measured variables. Changes in BPF between baseline and follow-up were found to be statistically significant (p < 0.001), suggesting a pathological change. Patients with low brain atrophy risk had a similar outcome regardless of DMT selection. In patients with high brain atrophy risk, high- and moderate-efficacy DMTs performed similarly, while a 2-fold worse BPF change was predicted for patients selecting low-efficacy DMTs (p < 0.001). Similar results were observed in a sensitivity analysis adjusting for pseudoatrophy effects in a sub-population of patients treated with natalizumab. ConclusionsThe relative benefit of selecting higher efficacy treatments may vary depending on patients’ baseline brain atrophy risk. Poor outcomes are predicted in individuals with high baseline risk who are treated with low-efficacy DMTs.

Lower or Higher Oxygenation Targets in Patients With COVID-19 in the ICU: A Secondary Bayesian Analysis of the Handling Oxygenation Targets in COVID-19 Trial

BackgroundIn the Handling Oxygenation Targets in COVID-19 (HOT-COVID) trial, a partial pressure of arterial oxygen (PaO2) target of 60 mmHg compared to 90 mmHg resulted in more days alive without life support at 90 days in adult ICU patients with coronavirus disease 2019 (COVID-19) and hypoxemia. The trial was stopped after enrolling 726 of 780 planned patients due to slow recruitment. Here, we present the preplanned Bayesian analysis of the HOT-COVID trial. Research QuestionWhat are the probabilities of any and of clinically relevant benefits from a PaO2 target of 60 mmHg versus 90 mmHg in adult ICU patients with COVID-19 and hypoxemia, and is there heterogeneity of treatment effects (HTE) according to selected baseline characteristics? Study Designand Methods We analyzed days alive without life support and 90-day mortality in the HOT-COVID intention-to-treat population (n=697) using Bayesian general linear models to assess probabilities for benefit or harm, including clinically relevant benefits defined as >1 day alive without life support and >2 percentage points lower 90-day mortality. HTE was evaluated based on baseline SOFA scores, PaO2:FiO2, norepinephrine doses, and lactate concentrations. ResultsThe mean difference in days alive without life support was 5.7 days (95% credible interval (CrI): 0.2 to 11.2), with a 95.2% probability of clinically relevant benefit and a 98.0% probability of any benefit from the lower PaO2 target. The risk difference in 90-day mortality was -4.6 percentage points (95% CrI: -11.8 to 2.6), with a 76.5% probability of a clinically relevant benefit from the lower target. HTE analyses revealed potential interaction with baseline norepinephrine dose and lactate concentrations for both outcomes. InterpretationIn ICU patients with COVID-19 and hypoxemia, we found high probability for a clinically relevant benefit of targeting a PaO2 of 60 mmHg vs. 90 mmHg on number of days alive without life support.

Effect of Aficamten on Health Status Outcomes in Obstructive Hypertrophic Cardiomyopathy: Results From SEQUOIA-HCM

BackgroundA primary goal in treating obstructive hypertrophic cardiomyopathy (oHCM) is to improve patients’ health status: their symptoms, function, and quality of life. The health status benefits of aficamten, a novel cardiac myosin inhibitor, have not been comprehensively described. ObjectivesThis study sought to determine the effect of aficamten on patient-reported health status, including symptoms of fatigue, shortness of breath, chest pain, physical and social limitations, and quality of life. MethodsSEQUOIA-HCM (Phase 3 Trial to Evaluate the Efficacy and Safety of Aficamten Compared to Placebo in Adults With Symptomatic oHCM) randomized symptomatic adults with oHCM to 24 weeks of aficamten (n = 142) or placebo (n = 140), followed by a 4-week washout. The Kansas City Cardiomyopathy Questionnaire (KCCQ) and Seattle Angina Questionnaire 7-item (SAQ7) were serially administered. Changes in mean KCCQ—Overall Summary Score (KCCQ-OSS) and SAQ7—Summary Score (SAQ7-SS) from baseline to 24 weeks and following treatment withdrawal were compared using linear regression adjusted for baseline scores and randomization strata. Proportions of patients with clinically important changes were compared. ResultsAmong 282 participants, the mean age was 59 ± 13 years, 115 (41%) were female, and 223 (79%) were White. Baseline KCCQ-OSS (69.3 ± 20.1 vs 67.3 ± 18.8) and SAQ7-SS (72.0 ± 21.0 vs 72.4 ± 18.3) were similar between aficamten and placebo groups. Treatment with aficamten, compared with placebo, improved both the mean KCCQ-OSS (13.3 ± 16.3 vs 6.1 ± 12.6; mean difference: 7.9; 95% CI: 4.8-11.0; P < 0.001) and SAQ7-SS (11.6 ± 17.4 vs 3.8 ± 14.4; mean difference: 7.8; 95% CI: 4.7-11.0; P < 0.001) at 24 weeks, with benefits emerging within 4 weeks. No heterogeneity in treatment effect was found across subgroups. A much larger proportion of participants experienced a very large health status improvement (≥20 points) with aficamten vs placebo (KCCQ-OSS: 29.7% vs 12.4%, number needed to treat: 5.8; SAQ7-SS: 31.2% vs 13.9%, number needed to treat: 5.8). Participants’ health status worsened significantly more after withdrawal from aficamten than placebo (KCCQ-OSS: −16.2 ± 19.0 vs −3.0 ± 9.6; P < 0.001; SAQ7-SS: −17.4 ± 21.4 vs −2.5 ± 13.3), further confirming a causal effect of aficamten. ConclusionsIn patients with symptomatic oHCM, treatment with aficamten resulted in markedly improved health status, including significant improvement in chest pain-related health status, than placebo. (Phase 3 Trial to Evaluate the Efficacy and Safety of Aficamten Compared to Placebo in Adults With Symptomatic oHCM [SEQUOIA-HCM]; NCT05186818)

Regional Heterogeneity of the Results of Glucagon-Like Peptide 1 Receptor Agonist Trials in Type 2 Diabetes: A Reanalysis of Individual Participant Data.

Multiregional trials are designed under the assumption that treatment effect applies to the entire target population, yet several factors may introduce geographic heterogeneity in treatment effect. We explored whether such variations exist in trials assessing the efficacy of glucagon-like peptide 1 receptor agonists (GLP-1RAs) in major cardiovascular events (MACE) in type 2 diabetes. A systematic search of Medline and the Cochrane Library was conducted from inception until 30 June 2020. We included international randomized controlled trials comparing any GLP-1RA versus placebo, with MACE as a primary end point. Individual participant data were subsequently requested from the sponsor or through data sharing platforms. For each trial, we calculated hazard ratios (HRs) and their 95% CIs for MACE, subgrouped by region. We then performed a random-effects meta-analysis and conducted meta-regressions to assess the influence of predetermined variables of interest on treatment effect. We included six trials including 45,426 patients. Baseline risk of MACE ranged from 2.9 per 100 patient-years in Southern Asia to 7.4 per 100 patient-years in Sub-Saharan Africa. HRs for MACE ranged between 0.25 (95% CI 0.05, 1.12) in Northern Africa to 0.98 (0.79, 1.22) in Western Europe. There was no significant subgroup difference across regions (P = 0.70). Baseline risk of MACE and indexes of development status (i.e., Human Development Index, gross domestic product) were independently associated with GLP-1RA efficacy. This study does not suggest any regional heterogeneity of GLP-1RA efficacy in MACE. However, a higher baseline risk and lower development status were associated with a greater benefit of these drugs.

Motivating for environmental protection: Evidence from county officials in China

This paper investigates the impact of changes in the officials’ performance evaluation system on ecological environment quality, as well as the effect of ecological improvements on officials’ promotion prospects. Analyses are conducted using the changes in the officials’ evaluation mechanism in China’s ecological function counties in 2013 as a quasi-natural experiment, incorporating a newly developed set of county-level comprehensive ecological environment index data and manually collated data on county leaders. Multiple methods are employed to address issues such as sample selection bias, reverse causality, and heterogeneity of treatment effects. The empirical analysis shows that the adjustment of officials’ performance evaluation indicators contributes to the improvement of ecological environment quality. This effect is more pronounced in regions where GDP evaluation is abolished, regions with lower economic development but better ecological environment foundations, regions closer to provincial capitals, and officials with shorter tenure, younger age, and male gender. The rationale behind this policy is that post-reform improvements in the ecological environment will increase the promotion prospects of officials in ecological function counties, enable these areas to secure additional ecological transfer payments, thereby enhancing their capacity for environmental expenditure, and establish a credible commitment mechanism for central-local contracts. This study not only examines the relationships between political incentives for officials and environmental protection, but also enriches the literature on environmental decentralization, multi-target governance, and environmental political economy.

Revisiting the effect of democracy on population health

Abstract I use a novel dichotomous measure of democracy to simulate a quasi-natural experiment and implement a difference-in-differences analysis to identify the heterogeneous treatment effect of democracy on population health across countries from 1960 to 2010. To counteract potential sources of bias resulting from unparallel and stochastic trends between treated and control units, I adopt a principal components difference-in-differences estimator that exploits factor proxies constructed from control units to account for unobserved trends. The main results indicate that countries that transitioned from non-democracy to democracy are more likely to experience health improvements, compared to countries retaining non-democratic institutions. However, the health-enhancing impact of democratization turns out to be much smaller in size than previously established. I posit that conventional estimates exaggerate the economic significance of the health returns to democratization due to inadequate attention to cross-border spillovers, global common shocks, and worldwide heterogeneity in the democracy-health nexus.

A Primer on Deep Learning for Causal Inference

This primer systematizes the emerging literature on causal inference using deep neural networks under the potential outcomes framework. It provides an intuitive introduction to building and optimizing custom deep learning models and shows how to adapt them to estimate/predict heterogeneous treatment effects. It also discusses ongoing work to extend causal inference to settings where confounding is nonlinear, time-varying, or encoded in text, networks, and images. To maximize accessibility, we also introduce prerequisite concepts from causal inference and deep learning. The primer differs from other treatments of deep learning and causal inference in its sharp focus on observational causal estimation, its extended exposition of key algorithms, and its detailed tutorials for implementing, training, and selecting among deep estimators in TensorFlow 2 and PyTorch.

Assessing homogeneous and heterogeneous economic impacts of fall armyworm management practices on farm performance in Ghana

One of the emerging challenges impinging on sustainable food production in sub-Saharan Africa is the invasion of the fall armyworm (FAW) pest. Data collected from farm households in different agro-ecological zones in Ghana and a Multivalued Treatment Effect (MVTE) model were used to argue that FAW management practices are key to stemming the debilitating effects of FAW infestations on farm performance. Previous studies have analysed homogeneous treatment effects to explain FAW management practices. The present study employs heterogeneous treatment effects to account for the differential effects of FAW management practices such as early planting, application of prescribed pesticides, and a combination of both practices while accounting for socioeconomic characteristics. Findings from the study reveal that distance to extension services exerts negative effects on adoption of early planting only, and adoption of both FAW management practices. Moreover, past FAW infestations tend to exert a positive effect on adoption of pesticide application only and adoption of a combination of the FAW management practices. Adopting the FAW management practices examined in this study assisted the maize producers in raising their farm performance. Socioeconomic characteristics also tend to influence the homogeneous treatment effects of adoption of the FAW management practices on farm performance. This finding indicates that heterogeneity within farm households is crucial for policy regarding adoption of FAW management practices.

Clonal hematopoiesis, cardiovascular events and treatment benefit in 63,700 individuals from five TIMI randomized trials.

Clonal hematopoiesis of indeterminate potential (CHIP) has been associated with an increased risk of cardiovascular (CV) disease in the general population. Currently, it is unclear whether this association is observed in large clinical trial cohorts with a high burden of existing CV disease or whether CV therapies can mitigate CHIP-associated CV risk. To address these questions, we studied 63,700 patients from five randomized trials that tested established therapies for CV disease, including treatments targeting the proteins PCSK9, SGLT2, P2Y12 and FXa. During a median follow-up of 2.5 years, 7,453 patients had at least one CV event (CV death, myocardial infarction (MI), ischemic stroke or coronary revascularization). The adjusted hazard ratio (aHR) for CV events for CHIP+ patients was 1.07 (95% CI: 0.99-1.16, P = 0.08), with consistent risk estimates across each component of CV risk. Significant heterogeneity in the risk of MI was observed, such that CHIP+ patients had a 30% increased risk of first MI (aHR = 1.31 (1.05-1.64), P = 0.02) but no increased risk of recurrent MI (aHR = 0.94 (0.79-1.13), Pint = 0.008), as compared to CHIP- patients. Moreover, no significant heterogeneity in treatment effect between individuals with and without CHIP was observed for any of the therapies studied in the five trials. These results indicate that in clinical trial populations, CHIP is associated with incident but not recurrent coronary events and that the presence of CHIP does not appear to identify patients who will derive greater benefit from commonly used CV therapies.

Is personalization of psychological pain treatments necessary? Evidence from a Bayesian variance ratio meta-analysis.

This is the first study to empirically determine the potential for data-driven personalization in the context of chronic primary pain (CPP). Effect sizes of psychological treatments for individuals with CPP are small to moderate on average. Aiming for better treatment outcomes for the individual patient, the call to personalize CPP treatment increased over time. However, empirical evidence that personalization of psychological treatments can optimize treatment outcomes in CPP is needed. This study seeks to estimate heterogeneity of treatment effect for cognitive behavioral therapy (CBT) as the psychological treatment approach for CPP with the greatest evidence base. For this purpose, a Bayesian variance ratio meta-regression is conducted using updated data from 2 recently published meta-analyses with randomized controlled trials comparing CBT delivered face-to-face to treatment-as-usual or waiting list controls. Heterogeneity in patients with CPP would be reflected by a larger overall variance in the post-treatment score compared with the control group. We found first evidence for an individual treatment effect in CBT compared with the control group. The estimate for the intercept was 0.06, indicating a 6% higher variance of end point values in the intervention groups. However, this result warrants careful consideration. Further research is needed to shed light on the heterogeneity of psychological treatment studies and thus to uncover the full potential of data-driven personalized psychotherapy for patients with CPP.A Bayesian variance ratio meta-regression indicates empirical evidence that data-driven personalized psychotherapy for patients with chronic primary pain could increase effects of cognitive behavioral therapy.

Ticagrelor monotherapy in ST-elevation myocardial infarction: An individual patient-level meta-analysis from TICO and T-PASS trials.

Patients with ST-elevation myocardial infarction (STEMI) tend to be excluded or under-represented in randomized clinical trials evaluating the effects of potent P2Y12 inhibitor monotherapy after short-term dual antiplatelet therapy (DAPT). Individual patient data were pooled from randomized clinical trials that included STEMI patients undergoing drug-eluting stent (DES) implantation and compared ticagrelor monotherapy after short-term (≤3months) DAPT versus ticagrelor-based 12-month DAPT in terms of centrally adjudicated clinical outcomes. The co-primary outcomes were efficacy outcome (composite of all-cause death, myocardial infarction, or stroke) and safety outcome (Bleeding Academic Research Consortium type 3 or 5 bleeding) at 1 year. The pooled cohort contained 2,253 patients with STEMI. The incidence of the primary efficacy outcome did not differ between the ticagrelor monotherapy group and the ticagrelor-based DAPT group (1.8% versus 2.0%; hazard ratio [HR]= 0.88; 95% confidence interval [CI]= 0.49-1.61; p= 0.684). There was no difference in cardiac death between the groups (0.6% versus 0.7%; HR= 0.89; 95% CI= 0.32-2.46; p= 0.822). The incidence of the primary safety outcome was significantly lower in the ticagrelor monotherapy group (2.3% versus 4.0%; HR= 0.56; 95% CI= 0.35-0.92; p= 0.020). No heterogeneity of treatment effects was observed for the primary outcomes across subgroups. In patients with STEMI treated with DES implantation, ticagrelor monotherapy after short-term DAPT was associated with lower major bleeding without an increase in the risk of ischemic events compared with ticagrelor-based 12-month DAPT. Further research is necessary to extend these findings to non-Asian patients. This study was funded by Biotronik (Bülach, Switzerland).

The impact of heterogeneity on the analysis of platform trials with normally distributed outcomes

BackgroundA platform trial approach allows adding arms to on-going trials to speed up intervention discovery programs. A control arm remains open for recruitment in a platform trial while intervention arms may be added after the onset of the study and could be terminated early for efficacy and/or futility when early stopping is allowed. The topic of utilising non-concurrent control data in the analysis of platform trials has been explored and discussed extensively. A less familiar issue is the presence of heterogeneity, which may exist for example due to modification of enrolment criteria and recruitment strategy.MethodWe conduct a simulation study to explore the impact of heterogeneity on the analysis of a two-stage platform trial design. We consider heterogeneity in treatment effects and heteroscedasticity in outcome data across stages for a normally distributed endpoint. We examine the performance of some hypothesis testing procedures and modelling strategies. The use of non-concurrent control data is also considered accordingly. Alongside standard regression analysis, we examine the performance of a novel method that was known as the pairwise trials analysis. It is similar to a network meta-analysis approach but adjusts for treatment comparisons instead of individual studies using fixed effects.ResultsSeveral testing strategies with concurrent control data seem to control the type I error rate at the required level when there is heteroscedasticity in outcome data across stages and/or a random cohort effect. The main parameter of treatment effects in some analysis models correspond to overall treatment effects weighted by stage wise sample sizes; while others correspond to the effect observed within a single stage. The characteristics of the estimates are not affected significantly by the presence of a random cohort effect and/ or heteroscedasticity.ConclusionIn view of heterogeneity in treatment effect across stages, the specification of null hypotheses in platform trials may need to be more subtle. We suggest employing testing procedure of adaptive design as opposed to testing the statistics from regression models; comparing the estimates from the pairwise trials analysis method and the regression model with interaction terms may indicate if heterogeneity is negligible.

Modern approaches for evaluating treatment effect heterogeneity from clinical trials and observational data.

In this paper, we review recent advances in statistical methods for the evaluation of the heterogeneity of treatment effects (HTE), including subgroup identification and estimation of individualized treatment regimens, from randomized clinical trials and observational studies. We identify several types of approaches using the features introduced in Lipkovich et al (Stat Med 2017;36: 136-196) that distinguish the recommended principled methods from basic methods for HTE evaluation that typically rely on rules of thumb and general guidelines (the methods are often referred to as common practices). We discuss the advantages and disadvantages of various principled methods as well as common measures for evaluating their performance. We use simulated data and a case study based on a historical clinical trial to illustrate several new approaches to HTE evaluation.

A Comprehensive Framework for Evaluating the Value Created by Real-World Evidence for Diverse Stakeholders: The Case for Coordinated Registry Networks.

This manuscript presents a comprehensive framework for the assessment of the value of real-world evidence (RWE) in healthcare decision-making. While RWE has been proposed to overcome some limitations of traditional, one-off studies, no systematic framework exists to measure if RWE actually lowers the burden. This framework aims to fill that gap by providing conceptual approaches for evaluating the time and cost efficiencies of RWE, thus guiding strategic investments in RWE infrastructure. The framework consists of four components: (114th Congress. 21st Century Cures Act.; 2015. https://www.congress.gov/114/plaws/publ255/PLAW-114publ255.pdf .) identification of stakeholders using and producing RWE, (National Health Council. Glossary of Patient Engagement Terms. Published 2019. Accessed May 18. 2021. https://nationalhealthcouncil.org/glossary-of-patient-engagement-terms/ .) understanding value propositions on how RWE can benefit stakeholders, (Center for Drug Evaluation and Research. CDER Patient-Focused Drug Development. U.S. Food & Drug Administration.) defining key performance indicators (KPIs), and (U.S. Department of Health and Human Services - Food and Drug Administration: Center for Devices and Radiological Health and Center for Biologics Evaluation and Research. Use of Real-World Evidence to Support Regulatory Decision-Making for Medical Devices - Guidance for Industry and Food and Drug Administration Staff. 2017. http://www.fda.gov/BiologicsBloodVaccines/GuidanceComplianceRegulatoryInformation/Guida .) establishing metrics and case studies to assess value. KPIs are categorized as 'better, faster, or cheaper" as an indicator of value: better focusing on high-quality actionable evidence; 'faster,' denoting time-saving in evidence generation, and 'cheaper,' emphasizing cost-efficiency decision compared to methodologies that do not involve data routinely collected in clinical practice. Metrics and relevant case studies are tailored based on stakeholder value propositions and selected KPIs that can be used to assess what value has been created by using RWE compared to traditional evidence-generation approaches and comparing different RWE sources. Operationalized through metrics and case studies drawn from the literature, the value of RWE is documented as improving treatment effect heterogeneity evaluation, expanding medical product labels, and expediting post-market compliance. RWE is also shown to reduce the cost and time required to produce evidence compared to traditional one-off approaches. An original example of a metric that measures the time saved by RWE methods to detect a signal of a product failure was presented based on analysis of the National Cardiovascular Disease Registry. The framework presented in this manuscript offers a comprehensive approach for evaluating the value of RWE, applicable to all stakeholders engaged in leveraging RWE for healthcare decision-making. Through the proposed metrics and illustrated case studies, valuable insights are provided into the heightened efficiency, cost-effectiveness, and improved decision-making within clinical and regulatory domains facilitated by RWE. While this framework is primarily focused on medical devices, it could potentially inform the determination of RWE value in other medical products. By discerning the variations in cost, time, and data utility among various evidence-generation methods, stakeholders are empowered to invest strategically in RWE infrastructure and shape future research endeavors.

Average treatment effects on binary outcomes with stochastic covariates.

When evaluating the effect of psychological treatments on a dichotomous outcome variable in a randomized controlled trial (RCT), covariate adjustment using logistic regression models is often applied. In the presence of covariates, average marginal effects (AMEs) are often preferred over odds ratios, as AMEs yield a clearer substantive and causal interpretation. However, standard error computation of AMEs neglects sampling-based uncertainty (i.e., covariate values are assumed to be fixed over repeated sampling), which leads to underestimation of AME standard errors in other generalized linear models (e.g., Poisson regression). In this paper, we present and compare approaches allowing for stochastic (i.e., randomly sampled) covariates in models for binary outcomes. In a simulation study, we investigated the quality of the AME and stochastic-covariate approaches focusing on statistical inference in finite samples. Our results indicate that the fixed-covariate approach provides reliable results only if there is no heterogeneity in interindividual treatment effects (i.e., presence of treatment-covariate interactions), while the stochastic-covariate approaches are preferable in all other simulated conditions. We provide an illustrative example from clinical psychology investigating the effect of a cognitive bias modification training on post-traumatic stress disorder while accounting for patients' anxiety using an RCT.

Outcome risk model development for heterogeneity of treatment effect analyses: a comparison of non-parametric machine learning methods and semi-parametric statistical methods

BackgroundIn randomized clinical trials, treatment effects may vary, and this possibility is referred to as heterogeneity of treatment effect (HTE). One way to quantify HTE is to partition participants into subgroups based on individual’s risk of experiencing an outcome, then measuring treatment effect by subgroup. Given the limited availability of externally validated outcome risk prediction models, internal models (created using the same dataset in which heterogeneity of treatment analyses also will be performed) are commonly developed for subgroup identification. We aim to compare different methods for generating internally developed outcome risk prediction models for subject partitioning in HTE analysis.MethodsThree approaches were selected for generating subgroups for the 2,441 participants from the United States enrolled in the ASPirin in Reducing Events in the Elderly (ASPREE) randomized controlled trial. An extant proportional hazards-based outcomes predictive risk model developed on the overall ASPREE cohort of 19,114 participants was identified and was used to partition United States’ participants by risk of experiencing a composite outcome of death, dementia, or persistent physical disability. Next, two supervised non-parametric machine learning outcome classifiers, decision trees and random forests, were used to develop multivariable risk prediction models and partition participants into subgroups with varied risks of experiencing the composite outcome. Then, we assessed how the partitioning from the proportional hazard model compared to those generated by the machine learning models in an HTE analysis of the 5-year absolute risk reduction (ARR) and hazard ratio for aspirin vs. placebo in each subgroup. Cochran’s Q test was used to detect if ARR varied significantly by subgroup.ResultsThe proportional hazard model was used to generate 5 subgroups using the quintiles of the estimated risk scores; the decision tree model was used to generate 6 subgroups (6 automatically determined tree leaves); and the random forest model was used to generate 5 subgroups using the quintiles of the prediction probability as risk scores. Using the semi-parametric proportional hazards model, the ARR at 5 years was 15.1% (95% CI 4.0–26.3%) for participants with the highest 20% of predicted risk. Using the random forest model, the ARR at 5 years was 13.7% (95% CI 3.1–24.4%) for participants with the highest 20% of predicted risk. The highest outcome risk group in the decision tree model also exhibited a risk reduction, but the confidence interval was wider (5-year ARR = 17.0%, 95% CI= -5.4–39.4%). Cochran’s Q test indicated ARR varied significantly only by subgroups created using the proportional hazards model. The hazard ratio for aspirin vs. placebo therapy did not significantly vary by subgroup in any of the models. The highest risk groups for the proportional hazards model and random forest model contained 230 participants each, while the highest risk group in the decision tree model contained 41 participants.ConclusionsThe choice of technique for internally developed models for outcome risk subgroups influences HTE analyses. The rationale for the use of a particular subgroup determination model in HTE analyses needs to be explicitly defined based on desired levels of explainability (with features importance), uncertainty of prediction, chances of overfitting, and assumptions regarding the underlying data structure. Replication of these analyses using data from other mid-size clinical trials may help to establish guidance for selecting an outcomes risk prediction modelling technique for HTE analyses.

The effect of vertical identification card laws on teenage tobacco and alcohol use.

We study the impact of vertical identification card laws, which changed the orientation of driver's licenses and state identification cards from horizontal to vertical for those under 21years, on teenage tobacco and alcohol use. We study this question using four national datasets (pooled national and state Youth Risk Behavior Surveillance System, National Youth Tobacco Survey, Current Population Survey to Tobacco Use Supplements, and Behavioral Risk Factor Surveillance System). We improve previous databases of vertical ID law implementation by using original archival research to identify the exact date of the law change. We estimate models using standard two-way fixed effects and stacked difference-in-differences that avoid bias from dynamic and heterogeneous treatment effects. Using data through 2021, we do not find evidence of reductions in teenage tobacco and alcohol use. While these laws reduce retail-based purchasing, they also increase social sourcing, thus leading to no net impact on use.

Impact of withholding early parenteral nutrition on 2-year mortality and functional outcome in critically ill adults.

In critically ill adults, withholding parenteral nutrition until 1week after intensive care admission (Late-PN) facilitated recovery as compared with early supplementation of insufficient enteral nutrition with parenteral nutrition (Early-PN). However, the impact on long-term mortality and functional outcome, in relation to the estimated nutritional risk, remains unclear. In this prospective follow-up study of the multicenter EPaNIC randomized controlled trial, we investigated the impact of Late-PN on 2-year mortality (N = 4640) and physical functioning, assessed by the 36-Item Short Form Health Survey (SF-36; in 3292 survivors, responding 819 [738-1058] days post-randomization). To account for missing data, we repeated the analyses in two imputed models. To identify potential heterogeneity of treatment effects, we investigated the impact of Late-PN in different nutritional risk subgroups as defined by Nutritional Risk Screening-2002-score, modified NUTrition Risk in the Critically Ill-score, and age (above/below 70years), and we evaluated whether there was statistically significant interaction between classification to a nutritional risk subgroup and the effect of the randomized intervention. Secondary outcomes were SF-36-derived physical and mental component scores (PCS & MCS). Two-year mortality (20.5% in Late-PN, 19.8% in Early-PN; P = 0.54) and physical functioning (70 [40-90] in both study-arms; P = 0.99) were similar in both groups, also after imputation of missing physical functioning data. Likewise, Late-PN had no impact on 2-year mortality and physical functioning in any nutritional risk subgroup. PCS and MCS were similar in both groups. Late-PN did not alter 2-year survival and physical functioning in adult critically ill patients, independent of anticipated nutritional risk.

Sample Size Estimation Using a Partially Clustered Frailty Model for Biomarker-Strategy Designs With Multiple Treatments.

Biomarker-guided therapy is a growing area of research in medicine. To optimize the use of biomarkers, several study designs including the biomarker-strategy design (BSD) have been proposed. Unlike traditional designs, the emphasis here is on comparing treatment strategies and not on treatment molecules as such. Patients are assigned to either a biomarker-based strategy (BBS) arm, in which biomarker-positive patients receive an experimental treatment that targets the identified biomarker, or a non-biomarker-based strategy (NBBS) arm, in which patients receive treatment regardless of their biomarker status. We proposed a simulation method based on a partially clustered frailty model (PCFM) as well as an extension of Freidlin formula to estimate the sample size required for BSD with multiple targeted treatments. The sample size was mainly influenced by the heterogeneity of treatment effect, the proportion of biomarker-negative patients, and the randomization ratio. The PCFM is well suited for the data structure and offers an alternative to traditional methodologies.