Effects In Observational Studies Research Articles

Bootstrap bias correction for average treatment effects with inverse propensity weights

The estimated average treatment effect in observational studies is biased if the assumptions of ignorability and overlap are not satisfied. To deal with this potential problem when propensity score weights are used in the estimation of the treatment effects, in this paper we propose a bootstrap bias correction estimator for the average treatment effect (ATE) obtained with the inverse propensity score (BBC-IPS) estimator. We show in simulations that the BBC-IPC performs well when we have misspecifications of the propensity score (PS) due to: omitted variables (ignorability property may not be satisfied), overlap (imbalances in distribution between treatment and control groups) and confounding effects between observables and unobservables (endogeneity). Further refinements in bias reductions of the ATE estimates in smaller samples are attained by iterating the BBC-IPS estimator.

Gain Scores Revisited: A Graphical Models Perspective

For misguided reasons, social scientists have long been reluctant to use gain scores for estimating causal effects. This article develops graphical models and graph-based arguments to show that gain score methods are a viable strategy for identifying causal treatment effects in observational studies. The proposed graphical models reveal that gain score methods rely on a bias-removing mechanism that is quite different to regular matching or covariance adjustment. While gain score methods offset noncausal associations via differencing, matching or covariance adjustment blocks noncausal association via conditioning. Since gain score estimators do not rely on conditioning, they are immune to measurement error in the pretest, bias amplification, and collider bias. The graph-based arguments also demonstrate that the key identifying assumption for gain score methods, the common trend assumption, is difficult to assess and justify when the pretest causally affects treatment assignment. Finally, we discuss the distinct role of pretests in the context of Lord’s paradox.

Assessing Treatment Effect Variation in Observational Studies: Results from a Data Challenge

A growing number of methods aim to assess the challenging question of treatment effect variation in observational studies. This special section of Observational Studies reports the results of a workshop conducted at the 2018 Atlantic Causal Inference Conference designed to understand the similarities and differences across these methods. We invited eight groups of researchers to analyze a synthetic observational data set that was generated using a recent large-scale randomized trial in education. Overall, participants employed a diverse set of methods, ranging from matching and flexible outcome modeling to semiparametric estimation and ensemble approaches. While there was broad consensus on the topline estimate, there were also large differences in estimated treatment effect moderation. This highlights the fact that estimating varying treatment effects in observational studies is often more challenging than estimating the average treatment effect alone. We suggest several directions for future work arising from this workshop.

Partial Least Squares Method for Treatment Effect in Observational Studies with Censored Outcomes

To estimate the true treatment effect on a censored outcome in observational studies, potential confounding effect and complex heterogeneity in the treatment assignment have to be properly adjusted. In this article, we demonstrate that the partial least squares method could be a valuable tool in this regard. It is showed that the partial least squares method not only can adequately account for the heterogeneity in treatment assignment, but also be robust to treatment assignment model misspecifications. Numerical results show that the partial least squares estimator is more efficient and robust. A real data set is analyzed to illustrate the proposed method.

Addressing missing data in confounders when estimating propensity scores for continuous exposures

Propensity score models are frequently used to estimate causal effects in observational studies. One unresolved issue in fitting these models is handling missing values in the propensity score model covariates. As these models usually contain a large set of covariates, using only individuals with complete data significantly decreases the sample size and statistical power. Several missing data imputation approaches have been proposed, including multiple imputation (MI), MI with missingness pattern (MIMP), and treatment mean imputation. Generalized boosted modeling (GBM), which is a nonparametric approach to estimate propensity scores, can automatically handle missingness in the covariates. Although the performance of MI, MIMP, and treatment mean imputation have previously been compared for binary treatments, they have not been compared for continuous exposures or with single imputation and GBM. We compared these approaches in estimating the generalized propensity score (GPS) for a continuous exposure in both a simulation study and in empirical data. Using GBM with the incomplete data to estimate the GPS did not perform well in the simulation. Missing values should be imputed before estimating propensity scores using GBM or any other approach for estimating the GPS.

Covariate association eliminating weights: a unified weighting framework for causal effect estimation.

SummaryWeighting methods offer an approach to estimating causal treatment effects in observational studies. However, if weights are estimated by maximum likelihood, misspecification of the treatment assignment model can lead to weighted estimators with substantial bias and variance. In this paper, we propose a unified framework for constructing weights such that a set of measured pretreatment covariates is unassociated with treatment assignment after weighting. We derive conditions for weight estimation by eliminating the associations between these covariates and treatment assignment characterized in a chosen treatment assignment model after weighting. The moment conditions in covariate balancing weight methods for binary, categorical and continuous treatments in cross-sectional settings are special cases of the conditions in our framework, which extends to longitudinal settings. Simulation shows that our method gives treatment effect estimates with smaller biases and variances than the maximum likelihood approach under treatment assignment model misspecification. We illustrate our method with an application to systemic lupus erythematosus data.

Are the effects of drugs to prevent and to treat heart failure always concordant? The statin paradox and its implications for understanding the actions of antidiabetic medications

Most treatments for chronic heart failure are effective both in preventing its onset and reducing its progression. However, statins prevent the development of heart failure, but they do not decrease morbidity and mortality in those with established heart failure. This apparent discordance cannot be explained by an effect to prevent interval myocardial infarctions. Instead, it seems that the disease that statins were preventing in trials of patients with a metabolic disorder was different from the disease that they were treating in trials of chronic heart failure. The most common phenotype of heart failure in patients with obesity and diabetes is heart failure with a preserved ejection fraction (HFpEF). In this disorder, the anti-inflammatory effects of statins might ameliorate myocardial fibrosis and cardiac filling abnormalities, but these actions may have little relevance to patients with heart failure and a reduced ejection fraction (HFrEF), whose primary derangement is cardiomyocyte loss and stretch. These distinctions may explain why statins were ineffective in trials that focused on HFrEF, but have been reported to produce favourable effects in observational studies of HFpEF. Similarly, selective cytokine antagonists were ineffective in HFrEF, but have been associated with benefits in HFpEF. These observations may have important implications for our understanding of the effects of antihyperglycaemic medications. Glucagon-like peptide-1 receptor agonists have had neutral effects on heart failure events in people at risk for HFpEF, but have exerted deleterious actions in HFrEF. Similarly, sodium-glucose co-transporter 2 inhibitors, which exert anti-inflammatory effects and reduce heart failure events in patients who are prone to HFpEF, may not be effective in HFrEF. The distinctions between HFrEF and HFpEF may explain why the effects of drugs on heart failure events in diabetes trials may not be relevant to their use in patients with systolic dysfunction.

Covariate balancing propensity score for a continuous treatment: Application to the efficacy of political advertisements

Propensity score matching and weighting are popular methods when estimating causal effects in observational studies. Beyond the assumption of unconfoundedness, however, these methods also require the model for the propensity score to be correctly specified. The recently proposed covariate balancing propensity score (CBPS) methodology increases the robustness to model misspecification by directly optimizing sample covariate balance between the treatment and control groups. In this paper, we extend the CBPS to a continuous treatment. We propose the covariate balancing generalized propensity score (CBGPS) methodology, which minimizes the association between covariates and the treatment. We develop both parametric and nonparametric approaches and show their superior performance over the standard maximum likelihood estimation in a simulation study. The CBGPS methodology is applied to an observational study, whose goal is to estimate the causal effects of political advertisements on campaign contributions. We also provide open-source software that implements the proposed methods.

Sensitivity analysis for mistakenly adjusting for mediators in estimating total effect in observational studies

ObjectivesIn observational studies, epidemiologists often attempt to estimate the total effect of an exposure on an outcome of interest. However, when the underlying diagram is unknown and limited knowledge is...

Modelling cumulative exposure for inference about drug effects in observational studies.

To demonstrate a modelling approach that controls for time-invariant allocation bias in estimation of associations of outcome with drug exposure. We show that in a model that includes terms for both ever-exposure versus never-exposure and cumulative exposure, the parameter for ever-exposure represents the effect of time-invariant allocation bias, and the parameter for cumulative exposure represents the effect of the drug after adjustment for this unmeasured confounding. This assumes no stepwise effect of the drug on the event rate, no reverse causation, and no unmeasured time-varying confounders. We demonstrated this by modelling the effect of statins on cardiovascular disease, for which the true effect has been well characterised in randomised trials, using time-updated Cox regression models in a national cohort of Type 2 diabetes patients. The crude hazard ratio associated with ever-use of statins was 1.13 in a standard cohort analysis comparing exposed with unexposed person-time intervals. When ever-never use and cumulative exposure are modelled jointly, the effect of statins can be estimated from the cumulative exposure parameter (hazard ratio 0.97 per year of exposure, 95% CI 0.97 to 0.98). The ever-exposed term (hazard ratio 1.20, 1.16 to 1.23) in this model can be interpreted as estimating the allocation bias. Where stepwise effects on the risk of adverse events are unlikely, as for instance for effects on risk of cancer, joint modelling of ever-never and cumulative exposure can be used to study the effects of multiple drugs and to distinguish causal effects from confounding by allocation.

An Instrumental Variable Tree Approach for Detecting Heterogeneous Treatment Effects in Observational Studies

This study addresses the ubiquitous challenge of using big observational data to identify heterogeneous treatment effects. This problem arises in precision medicine, targeted marketing, personalized education, and many other environments. Identifying heterogeneous treatment effects presents several analytical challenges including high-dimensionality and endogeneity issues. We develop a new instrumental variable tree (IVT) approach that incorporates the instrumental variable (IV) method into a causal tree (CT) to correct for potential endogeneity biases that may exist in observational data. Our IVT approach partitions subjects into subgroups with similar treatment effects within subgroups and different treatment effects across subgroups. The estimated treatment effects are asymptotically consistent under a set of mild assumptions. Using simulated data, we show our approach has a better coverage rate and smaller mean-squared error than the conventional CT approach. We also demonstrate that an instrumental variable forest (IVF) constructed using IVTs has better accuracy and stratification than a generalized random forest (GRF). Finally, by applying the IVF approach to an empirical assessment of laparoscopic colectomy, we demonstrate the importance of accounting for endogeneity to make accurate comparisons of the heterogeneous effects of the treatment (teaching hospitals) and control (non-teaching hospitals) on different types of patients.

Propensity score analysis with partially observed covariates: How should multiple imputation be used?

Inverse probability of treatment weighting is a popular propensity score-based approach to estimate marginal treatment effects in observational studies at risk of confounding bias. A major issue when estimating the propensity score is the presence of partially observed covariates. Multiple imputation is a natural approach to handle missing data on covariates: covariates are imputed and a propensity score analysis is performed in each imputed dataset to estimate the treatment effect. The treatment effect estimates from each imputed dataset are then combined to obtain an overall estimate. We call this method MIte. However, an alternative approach has been proposed, in which the propensity scores are combined across the imputed datasets (MIps). Therefore, there are remaining uncertainties about how to implement multiple imputation for propensity score analysis: (a) should we apply Rubin’s rules to the inverse probability of treatment weighting treatment effect estimates or to the propensity score estimates themselves? (b) does the outcome have to be included in the imputation model? (c) how should we estimate the variance of the inverse probability of treatment weighting estimator after multiple imputation? We studied the consistency and balancing properties of the MIte and MIps estimators and performed a simulation study to empirically assess their performance for the analysis of a binary outcome. We also compared the performance of these methods to complete case analysis and the missingness pattern approach, which uses a different propensity score model for each pattern of missingness, and a third multiple imputation approach in which the propensity score parameters are combined rather than the propensity scores themselves (MIpar). Under a missing at random mechanism, complete case and missingness pattern analyses were biased in most cases for estimating the marginal treatment effect, whereas multiple imputation approaches were approximately unbiased as long as the outcome was included in the imputation model. Only MIte was unbiased in all the studied scenarios and Rubin’s rules provided good variance estimates for MIte. The propensity score estimated in the MIte approach showed good balancing properties. In conclusion, when using multiple imputation in the inverse probability of treatment weighting context, MIte with the outcome included in the imputation model is the preferred approach.

Combining Propensity Score and Random Coefficient Modelling as an Approach to Analyse Complex Longitudinal Data

ABSTRACT
 ObjectivesWe looked for an approach to analyze/visualize a set of repeated measures of renal laboratory data (eGFR [estimated Glomerular Filtration Rate] from an observational population-based data set) as safety parameters in a longitudinal design and calculate annual changes in different sub-cohorts. Previous meta-analyses had struggled to address this problem (due to poor data quality and strong heterogeneity in underlying historical studies) and previous large population-based observational studies had only looked into binary outcomes.
 Particular challenges lay (1) in the complexity of the data set with irregularly spaced observation points, (2) in the observational character of the data with associated bias and confounding by indication and co-medication and (3) in the change of lab method during the observation period.
 ResultsOut of a population base of 400.000 we analysed linked longitudinal data of more than 1000 eligible patients with over 1000.000 prescription records of index drug or co-prescription drugs. Data were provided by the Dundee University Health Informatics Centre (HIC).
 We addressed the differences in covariates (which typically can lead to biased estimates of treatment effects in observational studies) via individual propensity scores (to reduce this bias by balancing the covariates in the two groups) and a hierarchical modelling approach.
 A Random Coefficient Model (via proc mixed in SAS 9.3) proved a much more powerful statistical tool than analysis of covariance of the summary measure during follow-up, particularly as the latter approach is less efficient when applied to longitudinal data with missing data points and irregularly spaced repeated measures.Visualization was achieved with the SAS 9.3 GPLOT procedure combined with a spline function. The historical change in lab method was addressed via a conversion of lab results to an internationally recognized standard (IDMS aligned method).
 We were able to achieve plausible and more precise estimates of the annual decline in eGFR in the patient group of interest than previous attempts from other research publications. This led to a publication in a high profile journal.
 ConclusionOur approach of combining a Propensity Score and Random Coefficient Modelling was successful to answer a question in drug safety using repeated measurement data from a longitudinal observational population based data set. This approach may be useful for other research questions in Drug Safety or in Comparative Clinical Effectiveness Research for continuous outcome measures.

Estimating treatment effects in observational studies with both prevalent and incident cohorts.

Registry databases are increasingly being used for comparative effectiveness research in cancer. Such databases reflect the real-world patient population and physician practice, and thus are natural sources for comparing multiple treatment scenarios and the associated long-term clinical outcomes. Registry databases usually include both incident and prevalent cohorts, which provide valuable complementary information for patients with more recent diagnoses in the incident cohort as well as patients with long-term follow-up data in the prevalent cohort. However, utilizing such data to derive valid inference poses two major challenges: the data from a prevalent cohort are not random samples of the target population, and there may be substantial differences in the baseline characteristics of patients between treatment arms, which influences the decisions about treatment selection in both cohorts. In this article, we extend propensity score methodology to observational studies that involve both prevalent and incident cohorts, and assess the effectiveness of radiation therapy in SEER-Medicare patients diagnosed with stage IV breast cancer. Specifically, we utilize the incident cohort to estimate the propensity for receiving radiation therapy, and then combine data from both the incident and prevalent cohorts to estimate the effect of radiation therapy by adjusting for the propensity scores in the model. We evaluate the proposed method with simulations. We demonstrate that the proposed propensity score method simultaneously removes sampling bias and selection bias under several assumptions.

Propensity scores: a tool to help quantify treatment effects in observational studies.

Propensity scores: a tool to help quantify treatment effects in observational studies.

Treatment Effect Estimation with Data-Driven Variable Decomposition

One fundamental problem in causal inference is the treatment effect estimation in observational studies when variables are confounded. Control for confounding effect is generally handled by propensity score. But it treats all observed variables as confounders and ignores the adjustment variables, which have no influence on treatment but are predictive of the outcome. Recently, it has been demonstrated that the adjustment variables are effective in reducing the variance of the estimated treatment effect. However, how to automatically separate the confounders and adjustment variables in observational studies is still an open problem, especially in the scenarios of high dimensional variables, which are common in big data era. In this paper, we propose a Data-Driven Variable Decomposition (D$^2$VD) algorithm, which can 1) automatically separate confounders and adjustment variables with a data driven approach, and 2) simultaneously estimate treatment effect in observational studies with high dimensional variables. Under standard assumptions, we show experimentally that the proposed D$^2$VD algorithm can automatically separate the variables precisely, and estimate treatment effect more accurately and with tighter confidence intervals than the state-of-the-art methods on both synthetic data and real online advertising dataset.

Evaluating different strategies for estimating treatment effects in observational studies

ABSTRACTSince the introduction of the propensity score (PS), methods for estimating treatment effects with observational data have received growing attention in the literature. Recent research has added substantially to the number of available statistical approaches for controlling confounding in such analyses. However, researchers need guidance to decide on the optimal analytic strategy for any given scenario. To address this gap, we conducted simulations evaluating both well-established methods (regression, PS weighting, stratification, and matching) and more recently proposed approaches (tree-based methods, local control, entropy balancing, genetic matching, prognostic scoring). The simulation scenarios included tree-based and smooth regression models as true data-generation mechanisms. We evaluated an extensive number of analysis strategies combining different treatment choices and outcome models. Key findings include 1) the lack of a single best strategy across all potential scenarios; 2) the importance of appropriately addressing interactions in the treatment choice model and/or outcome model; and 3) a tree-structured treatment choice model and a polynomial outcome model with second-order interactions performed well. One limitation to this initial assessment is the lack of heterogeneous simulation scenarios allowing treatment effects to vary by patient.

Rotation curve fitting and its fatal attraction to cores in realistically simulated galaxy observations

We study the role of systematic effects in observational studies of the cusp-core problem under the minimum disc approximation using a suite of high-resolution (25-pc softening length) hydrodynamical simulations of dwarf galaxies. We mimic realistic kinematic observations and fit the mock rotation curves with two analytic models commonly used to differentiate cores from cusps in the dark matter distribution. We find that the cored pseudo-isothermal sphere (ISO) model is strongly favoured by the reduced $\chi^2_\nu$ of the fits in spite of the fact that our simulations contain cuspy Navarro-Frenk-White profiles (NFW). We show that even idealized measurements of the gas circular motions can lead to the incorrect answer if velocity underestimates induced by pressure support, with a typical size of order $\sim$5 km s$^{-1}$ in the central kiloparsec, are neglected. Increasing the spatial resolution of the mock observations leads to more misleading results because the inner region, where the effect of pressure support is most significant, is better sampled. Fits to observations with a spatial resolution of 100 pc (2 arcsec at 10 Mpc) favour the ISO model in 78-90 per cent of the cases, while at 800-pc resolution, 40-78 per cent of the galaxies indicate the fictitious presence of a dark matter core. The coefficients of our best-fit models agree well with those reported in observational studies; therefore, we conclude that NFW haloes can not be ruled out reliably from this type of analysis.

An evaluation of bias in propensity score-adjusted non-linear regression models.

Propensity score methods are commonly used to adjust for observed confounding when estimating the conditional treatment effect in observational studies. One popular method, covariate adjustment of the propensity score in a regression model, has been empirically shown to be biased in non-linear models. However, no compelling underlying theoretical reason has been presented. We propose a new framework to investigate bias and consistency of propensity score-adjusted treatment effects in non-linear models that uses a simple geometric approach to forge a link between the consistency of the propensity score estimator and the collapsibility of non-linear models. Under this framework, we demonstrate that adjustment of the propensity score in an outcome model results in the decomposition of observed covariates into the propensity score and a remainder term. Omission of this remainder term from a non-collapsible regression model leads to biased estimates of the conditional odds ratio and conditional hazard ratio, but not for the conditional rate ratio. We further show, via simulation studies, that the bias in these propensity score-adjusted estimators increases with larger treatment effect size, larger covariate effects, and increasing dissimilarity between the coefficients of the covariates in the treatment model versus the outcome model.

Propensity score matching and subclassification in observational studies with multi-level treatments.

In this article, we develop new methods for estimating average treatment effects in observational studies, in settings with more than two treatment levels, assuming unconfoundedness given pretreatment variables. We emphasize propensity score subclassification and matching methods which have been among the most popular methods in the binary treatment literature. Whereas the literature has suggested that these particular propensity-based methods do not naturally extend to the multi-level treatment case, we show, using the concept of weak unconfoundedness and the notion of the generalized propensity score, that adjusting for a scalar function of the pretreatment variables removes all biases associated with observed pretreatment variables. We apply the proposed methods to an analysis of the effect of treatments for fibromyalgia. We also carry out a simulation study to assess the finite sample performance of the methods relative to previously proposed methods.