Potential Outcomes Framework Research Articles

Causal inference with observational data in addiction research.

Randomized controlled trials (RCTs) are the gold standard for making causal inferences, but RCTs are often not feasible in addiction research for ethical and logistic reasons. Observational data from real‐world settings have been increasingly used to guide clinical decisions and public health policies. This paper introduces the potential outcomes framework for causal inference and summarizes well‐established causal analysis methods for observational data, including matching, inverse probability treatment weighting, the instrumental variable method and interrupted time‐series analysis with controls. It provides examples in addiction research and guidance and analysis codes for conducting these analyses with example data sets.

Causal inference for observational longitudinal studies using deep survival models

ObjectiveCausal inference for observational longitudinal studies often requires the accurate estimation of treatment effects on time-to-event outcomes in the presence of time-dependent patient history and time-dependent covariates. Materials and MethodsTo tackle this longitudinal treatment effect estimation problem, we have developed a time-variant causal survival (TCS) model that uses the potential outcomes framework with an ensemble of recurrent subnetworks to estimate the difference in survival probabilities and its confidence interval over time as a function of time-dependent covariates and treatments. ResultsUsing simulated survival datasets, the TCS model showed good causal effect estimation performance across scenarios of varying sample dimensions, event rates, confounding and overlapping. However, increasing the sample size was not effective in alleviating the adverse impact of a high level of confounding. In a large clinical cohort study, TCS identified the expected conditional average treatment effect and detected individual treatment effect heterogeneity over time. TCS provides an efficient way to estimate and update individualized treatment effects over time, in order to improve clinical decisions. DiscussionThe use of a propensity score layer and potential outcome subnetworks helps correcting for selection bias. However, the proposed model is limited in its ability to correct the bias from unmeasured confounding, and more extensive testing of TCS under extreme scenarios such as low overlapping and the presence of unmeasured confounders is desired and left for future work. ConclusionTCS fills the gap in causal inference using deep learning techniques in survival analysis. It considers time-varying confounders and treatment options. Its treatment effect estimation can be easily compared with the conventional literature, which uses relative measures of treatment effect. We expect TCS will be particularly useful for identifying and quantifying treatment effect heterogeneity over time under the ever complex observational health care environment.

MarZIC: A Marginal Mediation Model for Zero-Inflated Compositional Mediators with Applications to Microbiome Data.

Background: The human microbiome can contribute to pathogeneses of many complex diseases by mediating disease-leading causal pathways. However, standard mediation analysis methods are not adequate to analyze the microbiome as a mediator due to the excessive number of zero-valued sequencing reads in the data and that the relative abundances have to sum to one. The two main challenges raised by the zero-inflated data structure are: (a) disentangling the mediation effect induced by the point mass at zero; and (b) identifying the observed zero-valued data points that are not zero (i.e., false zeros). Methods: We develop a novel marginal mediation analysis method under the potential-outcomes framework to address the issues. We also show that the marginal model can account for the compositional structure of microbiome data. Results: The mediation effect can be decomposed into two components that are inherent to the two-part nature of zero-inflated distributions. With probabilistic models to account for observing zeros, we also address the challenge with false zeros. A comprehensive simulation study and the application in a real microbiome study showcase our approach in comparison with existing approaches. Conclusions: When analyzing the zero-inflated microbiome composition as the mediators, MarZIC approach has better performance than standard causal mediation analysis approaches and existing competing approach.

Trade-offs or synergies? Hybridity and sustainable performance of dairy cooperatives in India

• Analyses the effect of National Dairy Plan on Indian dairy cooperatives. • Conceptualizes dairy cooperatives as hybrid organizations. • Regression analysis using 2-stage augmented inverse probability weighted estimator based on potential outcomes framework. • The performance of dairy cooperatives is affected by their hybrid characteristics. • Indian dairy cooperatives are characterized by synergetic outcomes, not trade-offs. India has come a long way in achieving milk security since it attained independence in 1947. During the 1950s India had to import about 55,000 tons of milk powder. By the year 2018–19 it had become one of the largest producers of milk in the world. The credit for this goes to the successful promotion of dairy cooperatives (DCs) in India. However, milk productivity in the country has continued to lag behind global averages. To increase milk productivity in India, the National Dairy Plan (NDP) was launched in 2011–12. Limited research appears to have been conducted on the effect of NDP on DC performance. Therefore, this article analyzes the effect of NDP on DC performance by conceptualizing DCs as hybrid organizations. However, there is limited research on how the hybrid character of cooperatives affects their performance. Regression analysis using 2-stage doubly robust, augmented inverse probability weighted (AIPW) estimator based on the potential outcomes framework on DC data collected via surveys and secondary reveals that NDP is imbibing characteristics of hybrid organizations in Indian DCs. Hybridity is statistically significant after controlling for village-level and DC-level variables. Financial hybridity is significant for four dependent variables: unit profit, quality of milk, human capability-building and community support. Autonomy, the other measure of hybridity, is significant for two dependent variables: growth of sales and unit profit. In other words, Indian DCs with hybrid characteristics actually pursue three different kinds of objectives: a) maximization of market-based logics (pursuit of self-interest, economic efficiency and profit maximization); b) maximization of community-oriented logics (pursuit of values such democracy, solidarity and autonomy); and, c) maximization of public benefit (public-sector led local economic development). The above discussion shows that Indian DCs are characterized by synergetic outcomes and not by trade-offs which contrasts with theoretical literature.

Improving the design stage of air pollution studies based on wind patterns

A growing literature in economics and epidemiology has exploited changes in wind patterns as a source of exogenous variation to better measure the acute health effects of air pollution. Since the distribution of wind components is not randomly distributed over time and related to other weather parameters, multivariate regression models are used to adjust for these confounding factors. However, this type of analysis relies on its ability to correctly adjust for all confounding factors and extrapolate to units without empirical counterfactuals. As an alternative to current practices and to gauge the extent of these issues, we propose to implement a causal inference pipeline to embed this type of observational study within an hypothetical randomized experiment. We illustrate this approach using daily data from Paris, France, over the 2008–2018 period. Using the Neyman–Rubin potential outcomes framework, we first define the treatment of interest as the effect of North-East winds on particulate matter concentrations compared to the effects of other wind directions. We then implement a matching algorithm to approximate a pairwise randomized experiment. It adjusts nonparametrically for observed confounders while avoiding model extrapolation by discarding treated days without similar control days. We find that the effective sample size for which treated and control units are comparable is surprisingly small. It is however reassuring that results on the matched sample are consistent with a standard regression analysis of the initial data. We finally carry out a quantitative bias analysis to check whether our results could be altered by an unmeasured confounder: estimated effects seem robust to a relatively large hidden bias. Our causal inference pipeline is a principled approach to improve the design of air pollution studies based on wind patterns.

Pathway-specific population attributable fractions.

A population attributable fraction represents the relative change in disease prevalence that one might expect if a particular exposure was absent from the population. Often, one might be interested in what percentage of this effect acts through particular pathways. For instance, the effect of a sedentary lifestyle on stroke risk may be mediated by blood pressure, body mass index and several other intermediate risk factors. We define a new metric, the pathway-specific population attributable fraction (PS-PAF), for mediating pathways of interest. PS-PAFs can be informally defined as the relative change in disease prevalence from an intervention that shifts the distribution of the mediator to its expected distribution if the risk factor were eliminated, and sometimes more simply as the relative change in disease prevalence if the mediating pathway were disabled. A potential outcomes framework is used for formal definitions and associated estimands are derived via relevant identifiability conditions. Computationally efficient estimators for PS-PAFs are derived based on these identifiability conditions. Calculations are demonstrated using INTERSTROKE-an international case-control study designed to quantify disease burden attributable to a number of known causal risk factors. The applied results suggest that mediating pathways from physical activity through blood pressure, blood lipids and body size explain comparable proportions of stroke disease burden, but a large proportion of the disease burden due to physical inactivity may be explained by alternative pathways. PS-PAFs measure disease burden attributable to differing mediating pathways and can generate insights into the dominant mechanisms by which a risk factor affects disease at a population level.

P17. Prenatal Cannabis Exposure Moderates the Relationship Between Sleep Hours and Internalizing Problems: A Causal Inference Analysis of ABCD Data

A 2019 epidemiological survey reported that 5.4% of women used cannabis during their pregnancy, yet we do not understand if this exposure is causally related to child health. Prior studies suggest prenatal cannabis exposure is associated with poor sleep and higher internalizing problems in children. Therefore, using the potential outcomes framework and statistical causal inference methods applied to the Adolescent Brain and Cognitive Development (ABCD) study, it was hypothesized that prenatal cannabis exposure would moderate the relationship between sleep hours and internalizing problems.

Discovery of genes positively modulating treatment effect using potential outcome framework and Bayesian update

BackgroundThe recent explosion of cancer genomics provides extensive information about mutations and gene expression changes in cancer. However, most of the identified gene mutations are not clinically utilized. It remains uncertain whether the presence of a certain genetic alteration will affect treatment response. Conventional statistics have limitations for causal inferences and are hard to gain sufficient power in genomic datasets. Here, we developed and evaluated a C-search algorithm for searching the causal genes that maximize the effect of the treatment.MethodsThe algorithm was developed based on the potential outcome framework and Bayesian posterior update. The precision of the algorithm was validated using a simulation dataset. The algorithm was implemented to a cBioPortal dataset. The genes discovered by the algorithm were externally validated within CancerSCAN screening data from Samsung Medical Center.ResultsSimulation data analysis showed that the C-search algorithm was able to identify nine causal genes out of ten. The C-search algorithm shows the discovery rate rapidly increasing until the 1500 data instances. Meanwhile, the log-rank test shows a slower increase in performance. The C-search algorithm was able to suggest nine causal genes from the cBioPortal Metabric dataset. Treating the patients with the causal genes is associated with better survival outcome in both the cBioPortal dataset and the CancerSCAN dataset which is used for external validation.ConclusionsOur C-search algorithm demonstrated better performance to identify causal effects of the genes than multiple log-rank test analysis especially within a limited number of data. The result suggests that the C-search can discover the causal genes from various genetic datasets, where the number of samples is limited compared to the number of variables.

Identifying and Estimating Causal Moderation for Treated and Targeted Subgroups

Extant literature on moderation effects narrowly focuses on the average moderated treatment effect across the entire sample (AMTE). Missing is the average moderated treatment effect on the treated (AMTT) and other targeted subgroups (AMTS). Much like the average treatment effect on the treated (ATT) for main effects, the AMTS changes the target of inferences from the entire sample to targeted subgroups. Relative to the AMTE, the AMTS is identified under weaker assumptions and often captures more policy-relevant effects. We present a theoretical framework that introduces the AMTS under the potential outcomes framework and delineates the assumptions for causal identification. We then propose a generalized propensity score method as a tool to estimate the AMTS using weights derived with Bayes Theorem. We illustrate the results and differences among the estimands using data from the Early Childhood Longitudinal Study. We conclude with suggestions for future research.

Causal Spillover Effects Using Instrumental Variables

I set up a potential outcomes framework to analyze spillover effects using instrumental variables. I characterize the population compliance types in a setting in which spillovers can occur on both treatment take-up and outcomes, and provide conditions for identification of the marginal distribution of compliance types. I show that intention-to-treat (ITT) parameters aggregate multiple direct and spillover effects for different compliance types, and hence do not have a clear link to causally interpretable parameters. Moreover, rescaling ITT parameters by first-stage estimands generally recovers a weighted combination of average effects where the sum of weights is larger than one. I then analyze identification of causal direct and spillover effects under one-sided noncompliance, and show that causal effects can be estimated by 2SLS in this case. I illustrate the proposed methods using data from an experiment on social interactions and voting behavior. I also introduce an alternative assumption, independence of the peers’ types, that identifies parameters of interest under two-sided noncompliance by restricting the amount of heterogeneity in average potential outcomes. Supplementary material of this article will be available in online.

Testing socioeconomic status and family socialization hypotheses of alcohol use in young people: A causal mediation analysis.

The effect of socioeconomic status on adolescent substance abuse may be mediated by family socialization practices. However, traditional mediation analysis using a product or difference method is susceptible to bias when assumptions are not addressed. We aimed to use a potential outcomes framework to assess assumptions of exposure-mediator interaction and of no confounding of the results. We revisited a traditional mediation analysis with a multiple mediator causal mediation approach using data from 17,761 Norwegian young people (13-18 years), 51% female. Data were collected through a print questionnaire. Socioeconomic status was operationalized as parental education and employment status (employed or receiving welfare); drinking behavior as the frequency of alcohol consumption and frequency of intoxication in the past year; and socialization practices as general parenting measures, alcohol-related parental permissiveness, and parent drinking behavior. There was no consistent evidence of exposure-mediator interaction. Formal sensitivity analysis of mediator-outcome confounding was not possible in the multiple mediator model, and this analysis supported the hypothesis that socioeconomic status effects on adolescent substance abuse are fully mediated by family socialization practices, with apparently stronger effects in younger age groups observed in plots. We found that the effect of socioeconomic status on adolescent substance abuse was fully mediated by family socialization practices. While our analysis provides more rigorous support for causal inferences than past work, we could not completely rule out the possibility of unmeasured confounding.

Selection bias and multiple inclusion criteria in observational studies

Abstract Objectives Spurious associations between an exposure and outcome not describing the causal estimand of interest can be the result of selection of the study population. Recently, sensitivity parameters and bounds have been proposed for selection bias, along the lines of sensitivity analysis previously proposed for bias due to unmeasured confounding. The basis for the bounds is that the researcher specifies values for sensitivity parameters describing associations under additional identifying assumptions. The sensitivity parameters describe aspects of the joint distribution of the outcome, the selection and a vector of unmeasured variables, for each treatment group respectively. In practice, selection of a study population is often made on the basis of several selection criteria, thereby affecting the proposed bounds. Methods We extend the previously proposed bounds to give additional guidance for practitioners to construct i) the sensitivity parameters for multiple selection variables and ii) an alternative assumption free bound, producing only logically feasible values. As a motivating example we derive the bounds for causal estimands in a study of perinatal risk factors for childhood onset Type 1 Diabetes Mellitus where selection of the study population was made by multiple inclusion criteria. To give further guidance for practitioners, we provide a data learner in R where both the sensitivity parameters and the assumption-free bounds are implemented. Results The assumption-free bounds can be both smaller and larger than the previously proposed bounds and can serve as an indicator of settings when the former bounds do not produce feasible values. The motivating example shows that the assumption-free bounds may not be appropriate when the outcome or treatment is rare. Conclusions Bounds can provide guidance in a sensitivity analysis to assess the magnitude of selection bias. Additional knowledge is used to produce values for sensitivity parameters under multiple selection criteria. The computation of values for the sensitivity parameters is complicated by the multiple inclusion/exclusion criteria, and a data learner in R is provided to facilitate their construction. For comparison and assessment of the feasibility of the bound an assumption free bound is provided using solely underlying assumptions in the framework of potential outcomes.

Economic analysis of the environmental sustainability of agriculture: recent studies using quasi-experimental methods

PurposeThe purpose of this review article is to demonstrate how the quasi-experimental approach has been used to study environmental and natural resource issues related to agricultural production.Design/methodology/approachThis review article first provides a short introduction to the quasi-experimental approach using the potential outcomes framework and then uses studies on the environmental sustainability of agricultural production to illustrate how quasi-experimental methods have been applied. Papers reviewed consist of studies that estimate the environmental externalities from agricultural production, evaluate agri-environmental and other related policies and programs, and demonstrate issues related to on-farm resource use and climate adaptation.FindingsDifference-in-differences (DID) and two-way fixed effects methods that utilize the spatial and temporal variation in panel data are widely used to estimate the causal impact of changes in agricultural production and policy on the environment. Utilizing the discontinuities and limits created by agricultural policies and regulations, local treatment effects on land and other input use are estimated using regression discontinuity (RD) or instrumental variable (IV) methods with cross-sectional data.Originality/valueChallenges faced by the food systems have made agricultural sustainability more critical than ever. Over the past three decades, the quasi-experimental approach has become the powerhouse of applied economic research. This review article focuses on quasi-experimental studies on the environmental sustainability of agriculture to provide methodological insights and to highlight gaps in the economics literature of agricultural sustainability.

Contrasting approaches for addressing non-adherence in randomized controlled trials: An illustration from the REFLUX trial.

In many randomized controlled trials, a substantial proportion of patients do not comply with the treatment protocol to which they have been randomly assigned. Randomized controlled trials are required to report results according to the intention-to-treat estimand, but recent methodological guidance recognizes the importance of estimating other causal quantities. This article outlines an analytical framework for randomized controlled trials with non-compliance. We apply the ICH E9 (R1) addendum and combine it with the potential outcomes framework to define key estimands, outline the major assumptions for identification of each estimand, and highlight the assumptions that cannot be verified from the randomized controlled trial data. We contrast the assumptions and estimates in a re-analysis of the REFLUX trial. We report alternative estimates for the effectiveness of receipt of laparoscopic surgery versus medical management for patients with gastro-intestinal reflux disease. The article finds that adjusted as-treated and per-protocol estimates were similar in magnitude to those based intention-to-treat methods. Instrumental variable estimates of the complier average causal effect were larger, with wider confidence intervals. We recommend that in randomized controlled trials with non-compliance, studies should outline which estimand is most relevant to the study context, evaluate key assumptions, and present estimates from a range of methods as a sensitivity analysis.

Testing for heterogeneity in the utility of a surrogate marker.

In studies that require long-term and/or costly follow-up of participants to evaluate a treatment, there is often interest in identifying and using a surrogate marker to evaluate the treatment effect. While several statistical methods have been proposed to evaluate potential surrogate markers, available methods generally do not account for or address the potential for a surrogate to vary in utility or strength by patient characteristics. Previous work examining surrogate markers has indicated that there may be such heterogeneity, that is, that a surrogate marker may be useful (with respect to capturing the treatment effect on the primary outcome) for some subgroups, but not for others. This heterogeneity is important to understand, particularly if the surrogate is to be used in a future trial to replace the primary outcome. In this paper, we propose an approach and estimation procedures to measure the surrogate strength as a function of a baseline covariate W and thus examine potential heterogeneity in the utility of the surrogate marker with respect to W. Within a potential outcome framework, we quantify the surrogate strength/utility using the proportion of treatment effect on the primary outcome that is explained by the treatment effect on the surrogate. We propose testing procedures to test for evidence of heterogeneity, examine finite sample performance of these methods via simulation, and illustrate the methods using AIDS clinical trialdata.

Inferring the causal effect of work zones on crashes: Methodology and a case study

The increasing number of crashes occurring in work zones has received considerable attention in recent years. Previous studies have mainly focused on associations between work zone configurations and crash occurrence. Although identification of associational relations helps us understand how work zones co-exist with crashes, it does not provide interventional guidelines necessary to improve safety of work zone operations. In this paper, a causal inference model based on the potential outcome framework is proposed to rigorously infer the causal effects of work zone presence on crash risks under various work zone configurations, along with robustness tests. In developing such a causal model, three research gaps are identified and addressed: (1) potential confounding bias due to unobservable roadway characteristics; (2) potential bias caused by unobserved variables in multisource data; and (3) lack of actually observed traffic data and weather information at the exact time when a crash occurred and lack of large-scale high-granular data. The proposed methodology is applied to 5,006 work zones in Pennsylvania from 2015 to 2017, and the results are validated via a series of robustness tests. The results show that the causal effect of a work zone on crash occurrence is significantly positive, especially on roadways with high traffic volumes, on long-distance work zones, and work zones conducted during daytime. It appears that conducting work zones during nighttime with the current deployment strategies on Pennsylvania state roads does not necessarily increase crash risks, but a work zone significantly increases crash risks during day time.

Noncompliance and Instrumental Variables for 2 K Factorial Experiments

Factorial experiments are widely used to assess the marginal, joint, and interactive effects of multiple concurrent factors. While a robust literature covers the design and analysis of these experiments, there is less work on how to handle treatment noncompliance in this setting. To fill this gap, we introduce a new methodology that uses the potential outcomes framework for analyzing factorial experiments with noncompliance on any number of factors. This framework builds on and extends the literature on both instrumental variables and factorial experiments in several ways. First, we define novel, complier-specific quantities of interest for this setting and show how to generalize key instrumental variables assumptions. Second, we show how partial compliance across factors gives researchers a choice over different types of compliers to target in estimation. Third, we show how to conduct inference for these new estimands from both the finite-population and superpopulation asymptotic perspectives. Finally, we illustrate these techniques by applying them to a field experiment on the effectiveness of different forms of get-out-the-vote canvassing. New easy-to-use, open-source software implements the methodology. Supplementary materials for this article are available online.

Sensitivity analysis of unmeasured confounding in causal inference based on exponential tilting and super learner

Causal inference under the potential outcome framework relies on the strongly ignorable treatment assumption. This assumption is usually questionable in observational studies, and the unmeasured confounding is one of the fundamental challenges in causal inference. To this end, we propose a new sensitivity analysis method to evaluate the impact of the unmeasured confounder by leveraging ideas of doubly robust estimators, the exponential tilt method, and the super learner algorithm. Compared to other existing methods of sensitivity analysis that parameterize the unmeasured confounder as a latent variable in the working models, the exponential tilting method does not impose any restrictions on the structure or models of the unmeasured confounders. In addition, in order to reduce the modeling bias of traditional parametric methods, we propose incorporating the super learner machine learning algorithm to perform nonparametric model estimation and the corresponding sensitivity analysis. Furthermore, most existing sensitivity analysis methods require multivariate sensitivity parameters, which make its choice difficult and subjective in practice. In comparison, the new method has a univariate sensitivity parameter with a nice and simple interpretation of log-odds ratios for binary outcomes, which makes its choice and the application of the new sensitivity analysis method very easy for practitioners.

Conformal Inference of Counterfactuals and Individual Treatment Effects

AbstractEvaluating treatment effect heterogeneity widely informs treatment decision making. At the moment, much emphasis is placed on the estimation of the conditional average treatment effect via flexible machine learning algorithms. While these methods enjoy some theoretical appeal in terms of consistency and convergence rates, they generally perform poorly in terms of uncertainty quantification. This is troubling since assessing risk is crucial for reliable decision-making in sensitive and uncertain environments. In this work, we propose a conformal inference-based approach that can produce reliable interval estimates for counterfactuals and individual treatment effects under the potential outcome framework. For completely randomized or stratified randomized experiments with perfect compliance, the intervals have guaranteed average coverage in finite samples regardless of the unknown data generating mechanism. For randomized experiments with ignorable compliance and general observational studies obeying the strong ignorability assumption, the intervals satisfy a doubly robust property which states the following: the average coverage is approximately controlled if either the propensity score or the conditional quantiles of potential outcomes can be estimated accurately. Numerical studies on both synthetic and real data sets empirically demonstrate that existing methods suffer from a significant coverage deficit even in simple models. In contrast, our methods achieve the desired coverage with reasonably short intervals.

A Unified Survey of Treatment Effect Heterogeneity Modelling and Uplift Modelling

A central question in many fields of scientific research is to determine how an outcome is affected by an action, i.e., to estimate the causal effect or treatment effect of an action. In recent years, in areas such as personalised healthcare, sociology, and online marketing, a need has emerged to estimate heterogeneous treatment effects with respect to individuals of different characteristics. To meet this need, two major approaches have been taken: treatment effect heterogeneity modelling and uplifting modelling. Researchers and practitioners in different communities have developed algorithms based on these approaches to estimate the heterogeneous treatment effects. In this article, we present a unified view of these two seemingly disconnected yet closely related approaches under the potential outcome framework. We provide a structured survey of existing methods following either of the two approaches, emphasising their inherent connections and using unified notation to facilitate comparisons. We also review the main applications of the surveyed methods in personalised marketing, personalised medicine, and sociology. Finally, we summarise and discuss the available software packages and source codes in terms of their coverage of different methods and applicability to different datasets, and we provide general guidelines for method selection.