Principal Stratification Framework Research Articles

A Bayesian Analysis of Two-Stage Randomized Experiments in the Presence of Interference, Treatment Nonadherence, and Missing Outcomes

Three critical issues for causal inference that often occur in modern, complicated experiments are interference, treatment nonadherence, and missing outcomes. A great deal of research efforts has been dedicated to developing causal inferential methodologies that address these issues separately. However, methodologies that can address these issues simultaneously are lacking. We propose a Bayesian causal inference methodology to address this gap. Our methodology extends existing causal frameworks and methods, specifically, two-staged randomized experiments and the principal stratification framework. In contrast to existing methods that invoke strong structural assumptions to identify principal causal effects, our Bayesian approach uses flexible distributional models that can accommodate the complexities of interference and missing outcomes, and that ensure that principal causal effects are weakly identifiable. We illustrate our methodology via simulation studies and a re-analysis of real-life data from an evaluation of India’s National Health Insurance Program. Our methodology enables us to identify new active causal effects that were not identified in past analyses. Ultimately, our simulation studies and case study demonstrate how our methodology can yield more informative analyses in modern experiments with interference, treatment nonadherence, missing outcomes, and complicated outcome generation mechanisms.

Principal stratification for quantile causal effects under partial compliance.

Within the principal stratification framework in causal inference, the majority of the literature has focused on binary compliance with an intervention and modelling means. Yet in some research areas, compliance is partial, and research questions-and hence analyses-are concerned with causal effects on (possibly high) quantiles rather than on shifts in average outcomes. Modelling partial compliance is challenging because it can suffer from lack of identifiability. We develop an approach to estimate quantile causal effects within a principal stratification framework, where principal strata are defined by the bivariate vector of (partial) compliance to the two levels of a binary intervention. We propose a conditional copula approach to impute the missing potential compliance and estimate the principal quantile treatment effect surface at high quantiles, allowing the copula association parameter to vary with the covariates. A bootstrap procedure is used to estimate the parameter to account for inflation due to imputation of missing compliance. Moreover, we describe precise assumptions on which the proposed approach is based, and investigate the finite sample behavior of our method by a simulation study. The proposed approach is used to study the 90th principal quantile treatment effect of executive stay-at-home orders on mitigating the risk of COVID-19 transmission in the United States.

Assessing treatment benefit in the presence of placebo response using the sequential parallel comparison design.

In clinical trials, placebo response is considered a beneficial effect arising from multiple factors, including the patient's expectations for the treatment. Its presence makes the classical parallel study design suboptimal and can bias the inference. The sequential parallel comparison design (SPCD), a two-stage design where the first stage is a classical parallel study design, followed by another parallel design among placebo subjects from the first stage, was proposed to address the shortcomings of the classical design. In SPCD, in lieu of treatment effect, a weighted average of the mean treatment difference in Stage I among all randomized subjects and the mean treatment difference in Stage II among placebo non-responders was proposed as the efficacy measure. However, by linking two possibly different populations, this weighted average lacks interpretability, and the choice of weight remains controversial. In this work, under the principal stratification framework, we propose a causal estimand for the treatment effect under each of three clinically important principal strata: Always Responders, Never Responders, and Drug-only Responders. To make the stratum treatment effect identifiable, we introduce a set of assumptions and two sensitivity parameters. By further considering the strata as latent characteristics, the sensitivity parameters can be estimated. An extensive simulation study is conducted to evaluate the operating characteristics of the proposed method. Finally, we apply our method on the ADAPT-A study data to assess the benefit of low-dose aripiprazole adjunctive to antidepressant therapy treatment.

Clarifying selection bias in cluster randomized trials.

In cluster randomized trials, patients are typically recruited after clusters are randomized, and the recruiters and patients may not be blinded to the assignment. This often leads to differential recruitment and consequently systematic differences in baseline characteristics of the recruited patients between intervention and control arms, inducing post-randomization selection bias. We aim to rigorously define causal estimands in the presence of selection bias. We elucidate the conditions under which standard covariate adjustment methods can validly estimate these estimands. We further discuss the additional data and assumptions necessary for estimating causal effects when such conditions are not met. Adopting the principal stratification framework in causal inference, we clarify there are two average treatment effect (ATE) estimands in cluster randomized trials: one for the overall population and one for the recruited population. We derive analytical formula of the two estimands in terms of principal-stratum-specific causal effects. Furthermore, using simulation studies, we assess the empirical performance of the multivariable regression adjustment method under different data generating processes leading to selection bias. When treatment effects are heterogeneous across principal strata, the average treatment effect on the overall population generally differs from the average treatment effect on the recruited population. A naïve intention-to-treat analysis of the recruited sample leads to biased estimates of both average treatment effects. In the presence of post-randomization selection and without additional data on the non-recruited subjects, the average treatment effect on the recruited population is estimable only when the treatment effects are homogeneous between principal strata, and the average treatment effect on the overall population is generally not estimable. The extent to which covariate adjustment can remove selection bias depends on the degree of effect heterogeneity across principal strata. There is a need and opportunity to improve the analysis of cluster randomized trials that are subject to post-randomization selection bias. For studies prone to selection bias, it is important to explicitly specify the target population that the causal estimands are defined on and adopt design and estimation strategies accordingly. To draw valid inferences about treatment effects, investigators should (1) assess the possibility of heterogeneous treatment effects, and (2) consider collecting data on covariates that are predictive of the recruitment process, and on the non-recruited population from external sources such as electronic health records.

Bayesian phase II clinical trial design with noncompliance.

Noncompliance issue is common in early phase clinical trials; and may lead to biased estimation of the intent-to-treat effect and incorrect conclusions for the clinical trial. In this work, we propose a Bayesian approach for sequentially monitoring the phase II randomized clinical trials that takes account for the noncompliance information. We adopt the principal stratification framework and propose to use Bayesian additive regression trees for selecting useful baseline covariates and estimating the complier average causal effect (CACE) for both efficacy and toxicity outcomes. The decision of early termination or not is then made adaptively based on the estimated CACE from the accumulated data. Simulation studies have confirmed the excellent performance of the proposed design in the presence of noncompliance.

Incorporating partial adherence into the principal stratification analysis framework

Participants in pragmatic clinical trials often partially adhere to treatment. However, to simplify the analysis, most studies dichotomize adherence (supposing that subjects received either full or no treatment), which can introduce biases in the results. For example, the popular approach of principal stratification is based on the concept that the population can be separated into strata based on how they will react to treatment assignment, but this framework does not include strata in which a partially adhering participant would belong. We expanded the principal stratification framework to allow partial adherers to have their own principal stratum and treatment level. The expanded approach is feasible in pragmatic settings. We have designed a Monte Carlo posterior sampling method to obtain the relevant parameter estimates. Simulations were completed under a range of settings where participants partially adhered to treatment, including a hypothetical setting from a published simulation trial on the topic of partial adherence. The inference method is additionally applied to data from a real randomized clinical trial that features partial adherence. Comparison of the simulation results indicated that our method is superior in most cases to the biased estimators obtained through standard principal stratification. Simulation results further suggest that our proposed method may lead to increased accuracy of inference in settings where study participants only partially adhere to assigned treatment.

Assessing Causal Effects of Extra Compulsory Learning on College Students’ Academic Performances

SummaryIn Italian state universities candidate freshmen must take an entrance examination. Candidates who obtain a test score less than or equal to a pre-fixed threshold may enrol at the university but must comply with an additional compulsory educational obligation, called obblighi formativi aggiuntivi (OFA). The OFA assignment rule appeals to a (sharp) regression discontinuity design with the entrance examination score acting as the forcing variable. We assess causal effects of OFA status by using data from a school of engineering of a specific Italian state university. For subpopulations of units for which our regression discontinuity design can be described as a local randomized experiment, we draw inference on the causal effects of OFA on students’ academic performances measured by using two variables: the number of university credits awarded at the end of the first academic year and the corresponding average grade. These outcome variables suffer from the problem of truncation by ‘death’, because neither is defined for students who decide not to enrol. Moreover, nor is the average grade defined for students who enrol but do not either take or pass any examination in the first academic year. We deal with these issues by using the framework of principal stratification and adopting a Bayesian approach to inference. We find some evidence that the receipt of OFA may negatively affect students’ academic performances, although the posterior distributions of the causal estimands have high variability.

Is Being an only Child Harmful to Psychological Health?: Evidence from an Instrumental Variable Analysis of China's one-child Policy

SummaryThe paper evaluates the effects of being an only child in a family on psychological health, leveraging data on the one-child policy in China. We use an instrumental variable approach to address the potential unmeasured confounding between the fertility decision and psychological health, where the instrumental variable is an index of the intensity of the implementation of the policy. We establish an analytical link between the local instrumental variable approach and principal stratification to accommodate the continuous instrumental variable. Within the principal stratification framework, we postulate a Bayesian hierarchical model to infer various causal estimands of policy interest while adjusting for the clustering data structure. We apply the method to the data from the China Family Panel Studies and find small but statistically significant negative effects of being an only child on self-reported psychological health for some subpopulations. Our analysis reveals treatment effect heterogeneity with respect to both observed and unobserved characteristics. In particular, urban males suffer the most from being only children, and the negative effect has larger magnitude if the families were more resistant to the one-child policy. We also conduct a sensitivity analysis to assess the key instrumental variable assumption.

Post-randomization Biomarker Effect Modification Analysis in an HIV Vaccine Clinical Trial.

While the HVTN 505 trial showed no overall efficacy of the tested vaccine to prevent HIV infection over placebo, markers measuring immune response to vaccination were strongly correlated with infection. This finding generated the hypothesis that some marker-defined vaccinated subgroups were partially protected whereas others had their risk increased. This hypothesis can be assessed using the principal stratification framework (Frangakis and Rubin, 2002) for studying treatment effect modification by an intermediate response variable, using methods in the sub-field of principal surrogate (PS) analysis that studies multiple principal strata. Unfortunately, available methods for PS analysis require an augmented study design not available in HVTN 505, and make untestable structural risk assumptions, motivating a need for more robust PS methods. Fortunately, another sub-field of principal stratification, survivor average causal effect (SACE) analysis (Rubin, 2006) - which studies effects in a single principal stratum - provides many methods not requiring an augmented design and making fewer assumptions. We show how, for a binary intermediate response variable, methods developed for SACE analysis can be adapted to PS analysis, providing new and more robust PS methods. Application to HVTN 505 supports that the vaccine partially protected individuals with vaccine-induced T-cells expressing certain combinations of functions.

Identifying and estimating principal causal effects in a multi-site trial of Early College High Schools

Randomized trials are often conducted with separate randomizations across multiple sites such as schools, voting districts, or hospitals. These sites can differ in important ways, including the site’s implementation quality, local conditions, and the composition of individuals. An important question in practice is whether—and under what assumptions—researchers can leverage this cross-site variation to learn more about the intervention. We address these questions in the principal stratification framework, which describes causal effects for subgroups defined by post-treatment quantities. We show that researchers can estimate certain principal causal effects via the multi-site design if they are willing to impose the strong assumption that the site-specific effects are independent of the site-specific distribution of stratum membership. We motivate this approach with a multi-site trial of the Early College High School Initiative, a unique secondary education program with the goal of increasing high school graduation rates and college enrollment. Our analyses corroborate previous studies suggesting that the initiative had positive effects for students who would have otherwise attended a low-quality high school, although power is limited.

Exploring Encouragement, Treatment, and Spillover Effects Using Principal Stratification, With Application to a Field Experiment on Teens’ Museum Attendance

This article revisits results from a field experiment, conducted in Florence, Italy, to study the effects of incentives provided to high school teens to motivate them to visit art museums. In the experiment, different classes of students were randomized to three types of encouragement and were offered a free visit to a main museum in the city. Using the principal stratification framework, the article explores causal pathways that may lead students to increase future visits, as induced by the encouragement received, or by the individual experience of the proposed free museum visit, or by the spillover of classmates’ experience. We do so by estimating and interpreting the causal effects of the three forms of encouragement within the principal strata defined by compliance behaviors. Bayesian inferential methods are used to derive the posterior distributions of weakly identified causal parameters.

Bayesian inference for a principal stratum estimand to assess the treatment effect in a subgroup characterized by postrandomization event occurrence.

The treatment effect in subgroups of patients is often of interest in randomized controlled clinical trials, as this may provide useful information on how to treat which patients best. When a specific subgroup is characterized by the absence of certain events that happen postrandomization, a naive analysis on the subset of patients without these events may be misleading. The principal stratification framework allows one to define an appropriate causal estimand in such settings. Statistical inference for the principal stratum estimand hinges on scientifically justified assumptions, which can be included with Bayesian methods through prior distributions. Our motivating example is a large randomized placebo-controlled trial of siponimod in patients with secondary progressive multiple sclerosis. The primary objective of this trial was to demonstrate the efficacy of siponimod relative to placebo in delaying disability progression for the whole study population. However, the treatment effect in the subgroup of patients who would not relapse during the trial is relevant from both a scientific and patient perspective. Assessing this subgroup treatment effect is challenging as there is strong evidence that siponimod reduces relapses. We describe in detail the scientific question of interest, the principal stratum estimand, the corresponding analysis method for binary endpoints, and sensitivity analyses. Although our work is motivated by a randomized clinical trial, the approach has broader appeal and could be adapted for observational studies.

Simultaneous Inference of Treatment Effect Modification by Intermediate Response Endpoint Principal Strata with Application to Vaccine Trials.

In randomized clinical trials, researchers are often interested in identifying an inexpensive intermediate study endpoint (typically a biomarker) that is a strong effect modifier of the treatment effect on a longer-term clinical endpoint of interest. Motivated by randomized placebo-controlled preventive vaccine efficacy trials, within the principal stratification framework a pseudo-score type estimator has been proposed to estimate disease risks conditional on the counter-factual biomarker of interest under each treatment assignment to vaccine or placebo, yielding an estimator of biomarker conditional vaccine efficacy. This method can be used for trial designs that use baseline predictors of the biomarker and/or designs that vaccinate disease-free placebo recipients at the end of the trial. In this article, we utilize the pseudo-score estimator to estimate the biomarker conditional vaccine efficacy adjusting for baseline covariates. We also propose a perturbation resampling method for making simultaneous inference on conditional vaccine efficacy over the values of the biomarker. We illustrate our method with datasets from two phase 3 dengue vaccine efficacy trials.

Evaluation of Language Training Programs in Luxembourg using Principal Stratification

In a world increasingly globalized, multiple language skills can create more employment opportunities. Several countries include language training programs in active labor market programs for the unemployed. We analyze the effects of a language training program on the re-employment probability and hourly wages of the unemployed simultaneously, using high-quality administrative data from Luxembourg. We address selection into training by exploiting the rich administrative information available, and account for the complication that wages are truncated by unemployment by adopting a principal stratification framework. Estimation is undertaken with a mixture model likelihood-based approach.To improve inference, we use the individual's hours worked as a secondary outcome and a stochastic dominance assumption. These two features considerably ameliorate the multimodality problem commonly encountered in mixture models. We also conduct sensitivity analysis to assess the unconfoundedness assumption employed. Our results strongly suggest a positive effect (of up to 12.7 percent) of the language training programs on the re-employment probability, but no effects on wages for those who are observed employed regardless of training participation. It appears that, in the context of an open and multilingual economy, language training improve employability but the language skills acquired are not sufficiently rewarded to be reflected in higher wages.

Income and substitution effects of a disability insurance reform

Disability insurance (DI) systems are widely criticized for their inherent work disincentives. This paper evaluates the effects of a Swiss DI reform that aims to lower DI benefits for a group of existing DI beneficiaries and introduces an additional level to the DI benefit schedule. The reform has only modest effects on earnings and employment but increases the disability degree of those threatened by a DI benefit decline. We estimate bounds on the income and substitution effects by employing the principal stratification framework. The income effect is quantitatively important, whereas the substitution effect is smaller and has bounds that include zero. The evidence suggests that caseworkers helped the insured with low labor market attachment to maintain a full DI benefit.

Principal ignorability in mediation analysis: through and beyond sequential ignorability

Author(s): Forastiere, L; Mattei, A; Ding, P | Abstract: In causal mediation analysis, the definitions of the natural direct and indirect effects involve potential outcomes that can never be observed, so-called a priori counterfactuals. This conceptual challenge translates into issues in identification, which requires strong and often unverifiable assumptions, including sequential ignorability. Alternatively, we can deal with post-treatment variables using the principal stratification framework, where causal effects are defined as comparisons of observable potential outcomes.We establish a novel bridge between mediation analysis and principal stratification, which helps to clarify and weaken the commonly used identifying assumptions for natural direct and indirect effects. Using principal stratification, we show how sequential ignorability extrapolates from observable potential outcomes to a priori counterfactuals, and propose alternative weaker principal ignorability-type assumptions. We illustrate the key concepts using a clinical trial.

Estimating causal effects of treatment in RCTs with provider and subject noncompliance

Subject noncompliance is a common problem in the analysis of randomized clinical trials (RCTs). With cognitive behavioral interventions, the addition of provider noncompliance further complicates making causal inference. As a motivating example, we consider an RCT of a motivational interviewing (MI)-based behavioral intervention for treating problem drug use. Treatment receipt depends on compliance of both a therapist (provider) and a patient (subject), where MI is received when the therapist adheres to the MI protocol and the patient actively participates in the intervention. However, therapists cannot be forced to follow protocol and patients cannot be forced to cooperate in an intervention. In this article, we (1) define a causal estimand of interest based on a principal stratification framework, the average causal effect of treatment among provider-subject pairs that comply with assignment or ACE(cc); (2) explore possible assumptions that identify ACE(cc); (3) develop novel estimators of ACE(cc); (4) evaluate estimators' statistical properties via simulation; and (5) apply our proposed methods for estimating ACE(cc) to data from our motivating example.

Evaluation of Language Training Programs in Luxembourg Using Principal Stratification

In a world increasingly globalized, multiple language skills can create more employment opportunities. Several countries include language training programs in active labor market programs for the unemployed. We analyze the effects of a language training program on the re-employment probability and hourly wages of the unemployed simultaneously, using highquality administrative data from Luxembourg. We address selection into training by exploiting the rich administrative information available, and account for the complication that wages are “truncated” by unemployment by adopting a principal stratification framework. Estimation is undertaken with a mixture model likelihood-based approach. To improve inference, we use the individual’s hours worked as a secondary outcome and a stochastic dominance assumption. These two features considerably ameliorate the multimodality problem commonly encountered in mixture models. We also conduct sensitivity analysis to assess the unconfoundedness assumption employed. Our results strongly suggest a positive effect (of up to 12.7 percent) of the language training programs on the re-employment probability, but no effects on wages for those who are observed employed regardless of training participation.

Bounding, An Accessible Method for Estimating Principal Causal Effects, Examined and Explained

ABSTRACTEstimating treatment effects for subgroups defined by posttreatment behavior (i.e., estimating causal effects in a principal stratification framework) can be technically challenging and heavily reliant on strong assumptions. We investigate an alternative path: using bounds to identify ranges of possible effects that are consistent with the data. This simple approach relies on fewer assumptions and yet can result in policy-relevant findings. As we show, even moderately predictive covariates can be used to substantially tighten bounds in a straightforward manner. Via simulation, we demonstrate which types of covariates are maximally beneficial. We conclude with an analysis of a multisite experimental study of Early College High Schools. When examining the program's impact on students completing the ninth grade “on-track” for college, we find little impact for ECHS students who would otherwise attend a high-quality high school, but substantial effects for those who would not. This suggests a potential benefit in expanding these programs in areas primarily served by lower quality schools.

Latent Class Survival Models Linked by Principal Stratification to Investigate Heterogenous Survival Subgroups Among Individuals With Early-Stage Kidney Cancer

ABSTRACTRates of kidney cancer have been increasing, with small incidental tumors experiencing the fastest growth rates. Much of the increase could be due to increased use of CT scans, MRIs, and ultrasounds for unrelated conditions. Many tumors might never have been detected or become symptomatic in the past. This suggests that many patients might benefit from less aggressive therapy, such as active surveillance by which tumors are surgically removed only if they become sufficiently large. However, it has been difficult for clinicians to identify subgroups of patients for whom treatment might be especially beneficial or harmful. In this work, we use a principal stratification framework to estimate the proportion and characteristics of individuals who have large or small hazard rates of death in two treatment arms. This allows us to assess who might be helped or harmed by aggressive treatment. We also use Weibull mixture models. This work differs from much previous work in that the survival classes upon which principal stratification is based are latent variables. That is, survival class is not an observed variable. We apply this work using Surveillance Epidemiology and End Results-Medicare claims data. Clinicians can use our methods for investigating treatments with heterogenous effects.