Biased Effect Estimates Research Articles

Who’s in and Who’s Out? Selection Bias in Aging Research

Selection bias presents a major threat to both internal and external validity in aging research. “Selection bias” refers to sample selection processes that lead to statistical associations in the study sample that are biased estimates of causal effects in the population of interest. These processes can lead to: (1) results that do not generalize to the population of interest (threat to external validity) or (2) biased effect estimates (associations that do not represent causal effects for any population, including the people in the sample; a threat to internal validity). In this presentation, we give an overview of selection bias in aging research. We will describe processes that can give rise to selection bias, highlight why they are particularly pervasive in this field, and present several examples of selection bias in aging research. We end with a brief summary of strategies to prevent and correct for selection bias in aging research.

Modeling Multivariate Count Time Series Data with a Vector Poisson Log-Normal Additive Model: Applications to Testing Treatment Effects in Single-Case Designs

In education and psychology, single-case designs (SCDs) have been used to detect treatment effects using time series data in the presence or absence of intervention. One popular design variant of SCDs is a multiple-baseline design for multiple outcomes, which often collects outcomes with some form of a count. A Poisson model is a natural choice for the count outcome. However, the assumption of the Poisson model that the outcome variable’s mean is equal to its variance is often violated in SCDs, as the variance is often larger than the mean (called overdispersion). In addition, when multiple outcomes are from the same participant, it is likely that they are correlated. In this paper, we present a vector Poisson log-normal additive (V-PLN-A) model to deal with (a) change processes (auto- and cross-correlations and data-driven trend) and (b) correlation and overdispersion in multivariate count time series. A multivariate normal distribution was adapted to account for correlation among multiple outcomes as well as possible overdispersion. The V-PLN-A model was applied to an educational intervention study to test treatment effects. Simulation study results showed that parameter recovery of the V-PLN-A model was satisfactory in a large number of timepoints using Bayesian analysis, and that ignoring change processes and overdispersion led to biased estimates of the treatment effects.

Statistical inference for association studies using electronic health records: handling both selection bias and outcome misclassification.

Health research using electronic health records (EHR) has gained popularity, but misclassification of EHR-derived disease status and lack of representativeness of the study sample can result in substantial bias in effect estimates and can impact power and type I error. In this paper, we develop new strategies for handling disease status misclassification and selection bias in EHR-based association studies. We first focus on each type of bias separately. For misclassification, we propose three novel likelihood-based bias correction strategies. A distinguishing feature of the EHR setting is that misclassification may be related to patient-varying factors, and the proposed methods leverage data in the EHR to estimate misclassification rates without gold standard labels. For addressing selection bias, we describe how calibration and inverse probability weighting methods from the survey sampling literature can be extended and applied to the EHRsetting. Addressing misclassification and selection biases simultaneously is a more challenging problem than dealing with each on its own, and we propose several new strategies. For all methods proposed, we derive valid standard error estimators and provide software for implementation. We provide a new suite of statistical estimation and inference strategies for addressing misclassification and selection bias simultaneously that is tailored to problems arising in EHR data analysis. We apply these methods to data from The Michigan Genomics Initiative, a longitudinal EHR-linkedbiorepository.

PCN23 Overall Survival Estimates Adjusting for Treatment Crossover in Patients Previously Untreated with an ALK Inhibitor with ALK+ NSCLC Using DATA from the Alta-1L Clinical Trial

Treatment crossover refers to the situation in randomised controlled trials (RCTs) where patients randomised to the control group switch to the experimental treatment. This leads to biased estimates of treatment effects if crossover is not appropriately controlled for. The objective of this study was to compare the overall survival (OS) from the ALTA-1L clinical trial derived from three different treatment switching adjustments with the unadjusted data. The marginal structural model (MSM), the inverse probability of censoring weights (IPCW) and the rank preserving structural failure time model (RPSFTM; with and without re-censoring) were utilized to attempt to adjust for the bias introduced from 61/138 patients switching from the crizotinib arm to the brigatinib arm in the ALTA-1L clinical trial (NCT02737501). The hazard ratio for brigatinib vs. crizotinib using the unadjusted OS data was 0.916 (95% CI: 0.572 – 1.466). Treatment switching methods estimated hazard ratios ranging from 0.446 to 0.939. Only RPSFTM including re-censoring increased the hazard ratio after removing the impact of subsequent brigatinib in the crizotinib arm, all other methods reduced the hazard ratio in favour of brigatinib. This study demonstrates that the choice of treatment switching methodology applied can have a large impact on the estimated treatment effect of brigatinib vs. crizotinib. Limitations associated with the analyses are: (1) immature data from ALTA-1L (24.1% and 26.8% OS events had occurred in the brigatinib and crizotinib arms, respectively), (2) the difficulty in validating requirements underpinning treatment switching methodologies and (3) other subsequent ALK inhibitor use was not accounted for.

Developing New Drugs for Mycobacterium tuberculosis Therapy: What Information Do We Get from Preclinical Animal Models?

Preclinical animal models of infection are employed to develop new agents but also to screen among molecules to rank them. There are often major differences between human pharmacokinetic (PK) profiles and those developed by animal models of infection, and these may lead to substantial differences in efficacy relative to that seen in humans. Linezolid is a repurposed agent employed to great effect for therapy of Mycobacterium tuberculosis In this study, we used the hollow-fiber infection model (HFIM) to evaluate the impact of different pharmacokinetic profiles of mice and nonhuman primates (NHP) versus humans on bacterial cell kill as well as resistance suppression. We examined both plasma and epithelial lining fluid (ELF) profiles. We examined simulated exposures equivalent to 600 mg and 900 mg daily of linezolid in humans. For both plasma and ELF exposures, the murine PK profile provided estimates of effect that were biased low relative to human and NHP PK profiles. Mathematical modeling identified a linkage between minimum concentrations (Cmin) and bacterial kill and peak concentrations (Cpeak) and resistance suppression, with the latter being supported by a prospective validation study. Finding new agents with novel mechanisms of action against M. tuberculosis is difficult. It would be a tragedy to discard a new agent because of a biased estimate of effect in a preclinical animal system. The HFIM provides a system to benchmark evaluation of new compounds in preclinical animal model systems against human PK effects (species scale-up estimates of PK), to safeguard against unwarranted rejection of promising new agents.

Epidemiologic Evidence of Glyphosate’s Carcinogenic Potential

Worldwide use of glyphosate has exploded since it was introduced in 1974. Most official assessments have disagreed with IARC’s 2015 conclusion that glyphosate is probably carcinogenic to humans (group 2A). In order to assess as directly as possible whether glyphosate is associated with an increased risk of non-Hodgkin lymphoma (NHL) in humans, we conducted a meta-analysis of the epidemiologic evidence with an a priori focus on high glyphosate exposures. We used six studies, adding the recently published Agricultural Health Study (AHS) results to five older case-control studies. To best address glyphosate’s carcinogenic potential, we focused on the workers in each study with the highest exposure by prioritizing in order: (1) highest cumulative exposure and longest lag, (2) highest cumulative exposure; (3) longest exposure duration and longest lag; (4) longest exposure duration; (5) longest latency or lag; and (6) ever-exposure. This quantification varied by study. We estimated the overall meta-relative risk of NHL to be 1.41 (95% confidence interval: 1.13-1.75). Due to our focus on the highest exposed workers, this is a stronger effect than previously reported. We compare and contrast our results with other meta-analyses and describe our concerns with the exposure assessment in the most recent AHS and its potential to bias the inference towards no increased risk. The AHS exposure group we used in our meta-analysis is likely to provide the least biased exposure effect estimate. Our findings support IARC’s assessment of glyphosate’s carcinogenic potential. This work has received much attention in the popular press and beyond, resulting in accusations of unethical scientific methods, public records requests and subpoenas.

Comparison of censoring assumptions to reduce bias in tuberculosis treatment cohort analyses.

ObjectiveObservational tuberculosis cohort studies are often limited by a lack of long-term data characterizing survival beyond the initial treatment outcome. Though Cox proportional hazards models are often applied to these data, differential risk of long-term survival, dependent on the initial treatment outcome, can lead to violations of model assumptions. We evaluate the performance of two alternate censoring approaches on reducing bias in treatment effect estimates.DesignWe simulate a typical multidrug-resistant tuberculosis cohort study and use Cox proportional hazards models to assess the relationship of an aggressive treatment regimen with hazard of death. We compare three assumptions regarding censored observations to determine which produces least biased treatment effect estimates: conventional non-informative censoring, an extension of short-term survival informed by literature, and incorporation of predicted long-term vital status.ResultsThe treatment regimen’s protective effect on death is consistently underestimated by the conventional censoring method, up to 7.6%. Models using the two alternative censoring techniques produce treatment effect estimates consistently stronger and less biased than the conventional method, underestimating the treatment effect by less than 2.4% across all scenarios.ConclusionsWhen model assumptions are violated, alternative censoring techniques can more accurately estimate associations between treatment and long-term survival. In multidrug-resistant tuberculosis cohort analyses, this bias reduction may yield more accurate and, larger effect estimates. This bias reduction can be achieved through use of standard statistical procedures with a simple re-coding of the censoring indicator.

Common Methods for Handling Missing Data in Marginal Structural Models: What Works and Why.

Marginal structural models (MSMs) are commonly used to estimate causal intervention effects in longitudinal nonrandomized studies. A common challenge when using MSMs to analyze observational studies is incomplete confounder data, where a poorly informed analysis method will lead to biased estimates of intervention effects. Despite a number of approaches described in the literature for handling missing data in MSMs, there is little guidance on what works in practice and why. We reviewed existing missing-data methods for MSMs and discussed the plausibility of their underlying assumptions. We also performed realistic simulations to quantify the bias of 5 methods used in practice: complete-case analysis, last observation carried forward, the missingness pattern approach, multiple imputation, and inverse-probability-of-missingness weighting. We considered 3 mechanisms for nonmonotone missing data encountered in research based on electronic health record data. Further illustration of the strengths and limitations of these analysis methods is provided through an application using a cohort of persons with sleep apnea: the research database of the French Observatoire Sommeil de la Fédération de Pneumologie. We recommend careful consideration of 1) the reasons for missingness, 2) whether missingness modifies the existing relationships among observed data, and 3) the scientific context and data source, to inform the choice of the appropriate method(s) for handling partially observed confounders in MSMs.

Assessing the risk of bias in choice of search sources for environmental meta‐analyses

Results of meta-analyses are potentially valuable for informing environmental policy and practice decisions. However, selective sampling of primary studies through searches exclusively using widely used bibliographic platform(s) could bias estimates of effect sizes. Such search strategies are common in environmental evidence reviews, and if risk of bias can be detected, this would provide the first empirical evidence that comprehensiveness of searches needs to be improved. We compare the impact of using single and multiple bibliographic platform(s) searches vs more comprehensive searches on estimates of mean effect sizes. We used 137 published meta-analyses, based on multiple source searches, analyzing 9388 studies: 8095 sourced from commercially published articles; and 1293 from grey literature and unpublished data. Single-platform and multiple-platform searches missed studies in 100 and 80 of the meta-analyses, respectively: 52 and 46 meta-analyses provided larger-effect estimates; 32 and 28 meta-analyses provided smaller-effect estimates; eight and four meta-analyses provided opposite direction of estimates; and two each were unable to estimate effects due to missing all studies. Further, we found significant positive log-linear relationships between proportions of studies missed and the deviations of mean effect sizes, suggesting that as the number of studies missed increases, deviation of mean effect size is likely to expand. We also found significant differences in mean effect sizes between indexed and non-indexed studies for 35% of meta-analyses, indicating high risk of bias when the searches were restricted. We conclude that the restricted searches are likely to lead to unrepresentative samples of studies and biased estimates of true effects.

Time-weighted average of fine particulate matter exposure and cause-specific mortality in China: a nationwide analysis

Most previous assessments of the hazardous effects attributable to fine particulate matter (PM2·5) exposure have used ambient PM2·5 as an exposure metric, resulting in substantial bias in effect estimates. We did a study to examine the association between cause-specific mortality and the time-weighted average of PM2·5 exposure after accounting for indoor exposure in 267 cities in China. We did a nationwide study, using Laser Egg air quality monitors in 36 cities to obtain data for indoor PM2·5 concentrations from 18 484 anonymised households between Nov 1, 2015 and July 2, 2018. We developed and validated a nationwide indoor PM2·5 prediction model for a further 302 cities by retrieving raw records of hourly concentrations from residents' air sensors; the model was used to predict indoor PM2·5 during 2013 to 2018. Daily ambient PM2·5 concentration data were estimated by averaging hourly ambient PM2·5 concentrations obtained from China's National Urban Air Quality Real-time Publishing Platform. Daily numbers of deaths from all non-accidental causes were obtained from 324 cities from the Disease Surveillance Point System of China between Jan 1, 2013, to Dec 31, 2017, and calculated for 267 cities that had an average daily mortality above three, and data for PM2·5 concentrations and meteorological information for at least 1 year between 2013 and 2017. We used distributed lag non-linear models to estimate city-specific associations between cause-specific mortality and reconstructed PM2·5 exposure by considering indoor PM2·5 exposure. We combined the city-specific effect estimates at the national level using a random effects meta-analysis. 13 972 records of daily indoor PM2·5 concentrations for 36 cities, extracted from 47 459 183 raw records from the sensors were included for modelling indoor PM2·5 levels. The nationwide indoor PM2·5 concentration was 40 μg/m3 (SD 21) between 2013 and 2017, which was approximately 20% lower than the ambient PM2·5 concentration of 50 μg/m3 (42). An increase of 10 μg/m3 in time-averaged PM2·5 exposure concentrations was associated with increased daily mortality estimates of 0·44% (95% CI 0·33-0·54) for total non-accidental causes, 0·50% (0·37-0·63) for cardiovascular diseases, 0·46% (0·28-0·63) for coronary heart disease, 0·49% (0·32-0·66) for stroke, 0·59% (0·39-0·79) for respiratory diseases, and 0·69% (0·45-0·92) for chronic obstructive pulmonary disease, respectively. Compared with previous estimations based on ambient PM2·5, our estimates approximately doubled the size of the effects related to PM2·5. This nationwide study revealed a higher mortality risk attributed to time-averaged indoor and ambient PM2·5 exposure compared with the risk associated with ambient PM2·5 exposure alone, which indicates that caution should be exercised when using ambient PM2·5 as a surrogate for PM2·5 exposure. National Natural Science Foundation of China (Youth Program) and the Fundamental Research Project of Beihang University.

Effect Direction Between Work-Family Conflict and Psychological Strain: A Replication and Extension

Research on the relationship between work-family conflict (WFC) and psychological strain provides largely inconsistent results about the direction of this effect. To identify a source of this variation, we review modeling approaches used in 15 studies derived from the only meta analytical test of the direction of this effect (Nohe et al., 2015), and 51 newer studies not yet included in this test. We find that none of these studies accounts for the confounding effect of unmeasured, trait-like between-person differences, for example, by adding random intercepts (RIs) to the structural model. Instead, they apply approaches that combine the within-person variance and the between-person variance (e.g., Crossed-Lagged Panel Model, CLPM), thereby leading to biased effect estimates. Recent methodological reviews have thus urged scholars to apply the RI-CLPM to solve this problem. Following these calls, our study replicates and extends prior research by modeling the effect of interest using the CLPM and the RI-CLPM. We seek to examine whether Nohe et al.’s (2015) result holds once the RI-CLPM is applied. We collected three-wave survey data from 12’241 working adults living in Germany. The data are representative of the German workforce and were collected annually from 2016 until 2018. The RI-CLPM yields a significantly better model fit than the CLPM. Whereas the results from our CLPM replicate the meta-analytical finding (i.e., WFC and psychological strain affect each other reciprocally), the RI-CLPM yields a different result: WFC leads to greater psychological strain and not vice versa. Implications for theory development and practice are discussed.

Epidural corticosteroid injections for lumbosacral radicular pain.

Epidural corticosteroid injections for lumbosacral radicular pain.

Propensity Score Analysis With Latent Covariates: Measurement Error Bias Correction Using the Covariate’s Posterior Mean, aka the Inclusive Factor Score

We address measurement error bias in propensity score (PS) analysis due to covariates that are latent variables. In the setting where latent covariate X is measured via multiple error-prone items W, PS analysis using several proxies for X—the W items themselves, a summary score (mean/sum of the items), or the conventional factor score (i.e., predicted value of X based on the measurement model)—often results in biased estimation of the causal effect because balancing the proxy (between exposure conditions) does not balance X. We propose an improved proxy: the conditional mean of X given the combination of W, the observed covariates Z, and exposure A, denoted [Formula: see text]. The theoretical support is that balancing [Formula: see text] (e.g., via weighting or matching) implies balancing the mean of X. For a latent X, we estimate [Formula: see text] by the inclusive factor score (iFS)—predicted value of X from a structural equation model that captures the joint distribution of [Formula: see text] given Z. Simulation shows that PS analysis using the iFS substantially improves balance on the first five moments of X and reduces bias in the estimated causal effect. Hence, within the proxy variables approach, we recommend this proxy over existing ones. We connect this proxy method to known results about valid weighting/matching functions. We illustrate the method in handling latent covariates when estimating the effect of out-of-school suspension on risk of later police arrests using National Longitudinal Study of Adolescent to Adult Health data.

Estimation of treatment effects following a sequential trial of multiple treatments.

When a clinical trial is subject to a series of interim analyses as a result of which the study may be terminated or modified, final frequentist analyses need to take account of the design used. Failure to do so may result in overstated levels of significance, biased effect estimates and confidence intervals with inadequate coverage probabilities. A wide variety of valid methods of frequentist analysis have been devised for sequential designs comparing a single experimental treatment with a single control treatment. It is less clear how to perform the final analysis of a sequential or adaptive design applied in a more complex setting, for example, to determine which treatment or set of treatments amongst several candidates should be recommended. This article has been motivated by consideration of a trial in which four treatments for sepsis are to be compared, with interim analyses allowing the dropping of treatments or termination of the trial to declare a single winner or to conclude that there is little difference between the treatments that remain. The approach taken is based on the method of Rao‐Blackwellization which enhances the accuracy of unbiased estimates available from the first interim analysis by taking their conditional expectations given final sufficient statistics. Analytic approaches to determine such expectations are difficult and specific to the details of the design: instead “reverse simulations” are conducted to construct replicate realizations of the first interim analysis from the final test statistics. The method also provides approximate confidence intervals for the differences between treatments.

Estimation of treatment effects and model diagnostics with two-way time-varying treatment switching: an application to a head and neck study.

Treatment switching frequently occurs in clinical trials due to ethical reasons. Intent-to-treat analysis without adjusting for switching yields biased and inefficient estimates of the treatment effects. In this paper, we propose a class of semiparametric semi-competing risks transition survival models to accommodate two-way time-varying switching. Theoretical properties of the proposed method are examined. An efficient expectation-maximization algorithm is derived to obtain maximum likelihood estimates and model diagnostic tools. Existing software is used to implement the algorithm. Simulation studies are conducted to demonstrate the validity of the model. The proposed method is further applied to data from a clinical trial with patients having recurrent or metastatic squamous-cell carcinoma of head and neck.

A Comparison of Matching and Weighting Methods for Causal Inference Based on Routine Health Insurance Data, or: What to do If an RCT is Impossible.

Due to a multitude of reasons Randomized Control Trials on the basis of so-called "routine data" provided by insurance companies cannot be conducted. Therefore the estimation of "causal effects" for any kind of treatment is hampered since systematic bias due to specific selection processes must be suspected. The basic problem of counterfactual, which is to evaluate the difference between two potential outcomes for the same unit, is discussed. The focus lies on the comparison of the performance of different approaches to control for systematic differences between treatment and control group. These strategies are all based on propensity scores, namely matching or pruning, IPTW (inverse probability treatment weighting) and entropy balancing. Methods to evaluate these strategies are presented. A logit model is employed with 87 predictors to estimate the propensity score or to estimate the entropy balancing weights. All analyses are restricted to estimate the ATT (Average Treatment Effect for the Treated) Exemplary data come from a prospective controlled intervention-study with two measurement occasions. Data contain 35 857 chronically ill insurants with diabetes, congestive heart failure, arteriosclerosis, coronary heart disease or hypertension of one German sickness fund. The intervention group was offered an individual telephone coaching to improve health behavior and slow down disease progression while the control group received treatment as usual. Randomization took place before the insurants' consent to participate was obtained so assumptions of an RCT are violated. A weighted mixture model (difference-in-difference) as the causal model of interest is employed to estimate treatment effects in terms of costs distinguishing the categories outpatient costs, medication costs, and total costs. It is shown that entropy balancing performs best with respect to balancing treatment and control group at baseline for the first three moments of all 87 predictors. This will result in least biased estimates of the treatment effect.

Estimating Willingness to Pay from Count Data When Survey Responses are Rounded

Recall data of visits to recreational sites often contain reported numbers that appear to be rounded to nearby focal points (e.g., the closest 5 or 10). Failure to address this rounding has been shown to produce biased estimates of marginal effects and non-linear combinations of coefficients such as willingness to pay. We investigate the relative performance of three count data models used with data of the kind typically found in survey data. We create a dataset based on observed recreational trip counts and associated trip costs that exhibits substantial rounding. We then conduct a Monte Carlo simulation exercise to compare the estimated parameters, willingness to pay, and the average consumer surplus per trip for three alternative estimators: a standard Poisson model with no adjustment for rounding, a censored Poisson model, a grouped Poisson model, and a latent class Poisson model with rounding and non-rounding classes. The standard Poisson model with no adjustment for rounding exhibits significant and persistent bias, especially in estimates of non-linear effects. The grouped and latent class Poisson models, in contrast, show little to no bias in estimates.

Nalmefene in alcohol‐dependent patients with a high drinking risk: A limited efficacy in reducing alcohol consumption

Nalmefene authorization in Europe for reducing alcohol consumption in alcohol-dependent patients with high or very high drinking risk level and maintaining a high drinking risk level 2 weeks after initial assessment was granted on the grounds of controversial data. There had been no randomized controlled trial (RCT) conducted in the target population at that point. The market authorization was based on data from subgroup analyses of the pivotal studies: ESENSE1 (NCT00811720), ESENSE2 (NCT00812461), and SENSE (NCT00811941). However, subgroup analyses are only exploratory. It is therefore reasonable to question if the new RCT assessing the efficacy of nalmefene in patients with a high drinking risk1 can add to the previous body of evidence. We argue that this study shares the weaknesses of the previous phase 3 trials that led to questioning the efficacy of nalmefene in reducing alcohol consumption.2 Overall, although the effect sizes observed on the consumption outcomes were higher than in the previous studies, the clinical relevance of such differences remains questionable. Regarding total alcohol consumption, the difference in the adjusted change from baseline between nalmefene 20 mg and placebo (expressed in least square mean) at 6 months was −11 g of alcohol/day (i.e., about a 100-mL glass of wine). It is also not known how much these differences were due to attrition bias. Compared to placebo (89.4%), the treatment completion rates were significantly lower in the nalmefene 20-mg (76.2%, P < 0.001) and nalmefene 10-mg (75.5%, P < 0.001) groups. Whereas a loss to follow-up of 5% or lower is usually of little concern, such a loss of >20% prevents good-quality intention-to-treat analysis and can cause biased estimates of the treatment effect.3, 4 The rate of discontinuation due to adverse events was higher in the groups receiving nalmefene than in the placebo group (18.1%, 18.5%, and 4.5%, respectively, for nalmefene 20-mg, nalmefene 10-mg, and placebo). These characteristics were already observed in a meta-analysis of the initial trials submitted to the European Medicine Agency.5 In addition, a primary outcome assessed at 12 weeks and a follow-up ending at 24 weeks is too short to evaluate a drug intending to reduce harm. No RCT has assessed the efficacy of nalmefene in the long term. Only one RCT has assessed the efficacy of nalmefene after 1 year of treatment, and showed limited and inconsistent efficacy on consumption outcomes.5 Currently, the harm-reduction approach is based on models that are basically an extrapolation of the results from subgroup analyses of the previous pivotal RCT beyond the trial time horizon.6 Moreover, it is unlikely that a slight reduction in consumption, even if sustained, could be clinically relevant considering that the model is dealing with patients who had experienced high consumption for a long time, which is different to subjects with excessive use. None of the previous trials demonstrated any benefit of nalmefene on harm related to alcohol use, and the current study failed to show significant effect of nalmefene on health-related quality of life (i.e., Alcohol Quality of Life Scale, Short Form – 36 Health Survey, and EuroQol - 5 Dimension) at 24 weeks. In several European countries, such as in Germany, agencies for efficiency in health care have concluded that studies of nalmefene showed no additional benefit for alcohol dependence compared to naltrexone, another opioid antagonist that was an earlier and less costly comparator therapy.7 A recent independent indirect comparison concluded a lack of advantage of nalmefene versus naltrexone.8 Long-term psychosocial interventions by trained professionals remain the cornerstone of successful care.9 Further, treatment would be a Sisyphean task and would demonize vulnerable people who cannot stop their problem drinking if no comprehensive public health policy for alcohol control is implemented. Finally, readers must be aware that a Californian winery, Ridge Vineyards Inc., is a subsidiary company of Otsuka Holdings Co., Ltd. In conclusion, it should be recalled that the clinical significance of a statistically significant difference on a surrogate outcome should be critically appraised. A.B. and F.N. served as unpaid experts for several taskforces for the French drug agency (Agence Nationale de Sécurité du Médicament et des Produits de Santé), none of which were related to the topic. A.B. was a late co-investigator for NCT01738282 trial (baclofen) but did not recruit. C.P. has no conflict of interest to declare.

Ten questions to consider when interpreting results of a meta-epidemiological study-the MetaBLIND study as a case.

Randomized clinical trials underpin evidence-based clinical practice, but flaws in their conduct may lead to biased estimates of intervention effects and hence invalid treatment recommendations. The main approach to the empirical study of bias is to collate a number of meta-analyses and, within each, compare the results of trials with and without a methodological characteristic such as blinding of participants and health professionals. Estimated within-meta-analysis differences are combined across meta-analyses, leading to an estimate of mean bias. Such "meta-epidemiological" studies are published in increasing numbers and have the potential to inform trial design, assessment of risk of bias, and reporting guidelines. However, their interpretation is complicated by issues of confounding, imprecision, and applicability. We developed a guide for interpreting meta-epidemiological studies, illustrated using MetaBLIND, a large study on the impact of blinding. Applying generally accepted principles of research methodology to meta-epidemiology, we framed 10 questions covering the main issues to consider when interpreting results of such studies, including risk of systematic error, risk of random error, issues related to heterogeneity, and theoretical plausibility. We suggest that readers of a meta-epidemiological study reflect comprehensively on the research question posed in the study, whether an experimental intervention was unequivocally identified for all included trials, the risk of misclassification of the trial characteristic, and the risk of confounding, i.e the adequacy of any adjustment for the likely confounders. We hope that our guide to interpretation of results of meta-epidemiological studies is helpful for readers of such studies.

Heterogeneous Treatment and Spillover Effects Under Clustered Network Interference

The bulk of causal inference studies rules out the presence of interference between units. However, in many real-world settings units are interconnected by social, physical or virtual ties and the effect of a treatment can spill from one unit to other connected individuals in the network. In these settings, interference should be taken into account to avoid biased estimates of the treatment effect, but it can also be leveraged to save resources and provide the intervention to a lower percentage of the population where the treatment is more effective and where the effect can spill over to other susceptible individuals. In fact, different people might respond differently not only to the treatment received but also to the treatment received by their network contacts. Understanding the heterogeneity of treatment and spillover effects can help policy-makers in the scale-up phase of the intervention, it can guide the design of targeting strategies with the ultimate goal of making the interventions more cost-effective, and it might even allow generalizing the level of treatment spillover effects in other populations. In this paper, we develop a machine learning method that makes use of tree-based algorithms and an Horvitz-Thompson estimator to assess the heterogeneity of treatment and spillover effects with respect to individual, neighborhood and network characteristics in the context of clustered network interference. We illustrate how the proposed binary tree methodology performs in a Monte Carlo simulation study. Additionally, we provide an application on a randomized experiment aimed at assessing the heterogeneous effects of information sessions on the uptake of a new weather insurance policy in rural China.