Quantitative falsification of instrumental variables assumption using balance measures.

Abstract

To the Editor: Instrumental variable analysis has been used to control for unmeasured confounding in nonrandomized studies.1–4 An instrumental variable (1) is associated with exposure, (2) affects outcome only through the exposure, and (3) is independent of confounders.1–4 If these key assumptions are satisfied (together with additional assumptions, such as homogeneity),1,3,4 instrumental variable analysis could consistently estimate the average causal effect of exposure.1,4 However, if one of the assumptions is violated, the estimate can be severely biased.1,3,4 Several methods are available for checking the first assumption,2,4 but there is no well-established method for checking the second and third assumptions. Some authors1,3 have argued that these assumptions are untestable, as they involve unmeasured confounding. Glymour et al5 suggested several approaches (eg, leverage prior causal assumptions) for evaluating the validity of instrumental variable, although, in certain situations, they might fail to identify a biased instrumental variable or inappropriately suggest that a valid instrumental variable is biased. In addition, balance of measured confounders between instrumental variable categories has been used as a supportive evidence for the third assumption.2,6 Alternatively, an imbalance of measured confounders can falsify this assumption. We propose the standardized difference (SDif), a robust balance measure used in propensity score methods,7,8 to falsify the third assumption by checking independence between an instrumental variable and measured confounders. If measured confounders are insufficiently balanced between instrumental variable categories, indicated by SDif values deviating from zero (eg, >0.10),7 this may also imply an imbalance of unmeasured confounders, even after conditioning on measured confounders (depending on the associations among instrumental variables and measured and unmeasured confounders). In that case, the third assumption is violated; hence, (un)adjusted instrumental variable analysis is inappropriate. However, if measured confounders are balanced, investigators should rely on background knowledge to argue that such balance could be carried over to unmeasured confounders.2,6 In a simulation study, we assessed the performance of SDif to quantitatively falsify the third assumption. In addition, we applied this measure in an empirical study on the relation between β2-agonist use and myocardial infarction, using physician preference as an instrumental variable. For details, we refer to the eAppendix (https://links.lww.com/EDE/A815). Key findings are summarized below. Data were generated with binary instrumental variable and exposure, continuous confounders (3 measured and 1 unmeasured), and a continuous outcome based on the causal diagram shown in the Figure (panel A). SDif was calculated for the measured confounders.FIGURE: Relation between bias and standardized difference of observed confounders between instrumental variable (IV) categories. Panel A shows a directed acyclic graph (DAG) of the simulations, where X = exposure, Y = outcome, C = observed confounders, U = unobserved confounder, and Z = IV. Panels B and C show the relation between the mean standardized difference versus bias of IV estimates (based on 10,000 simulations of 10,000 subjects). In panel B, observed confounders were not included in the IV models, whereas in panel C they were included. Observed confounders were adjusted for in conventional regression analysis in both panels B and C.Panel B shows the results of instrumental variable analysis without adjustment for measured confounders. The magnitude of bias in the instrumental variable estimate increased with decreasing balance of measured confounders between instrumental variable categories (eg, for an instrumental variable that was independent of unmeasured confounders, the bias ranged from 0.0 to 6.3 for corresponding SDif of 0.05–0.60). When the instrumental variable was independent of the measured confounders, but associated with the unmeasured confounders, instrumental variable estimates were biased, although the SDif was close to zero. Panel C shows the results of instrumental variable analysis with adjustment for measured confounders. When the instrumental variable was independent of the unmeasured confounder, effect estimates were unbiased. Moreover, the bias in adjusted instrumental variable estimates was smaller than that in unadjusted instrumental variable estimates. Importantly, when the instrumental variable was associated with measured and unmeasured confounders, estimates from adjusted instrumental variable models were more biased than those from conventional regression analysis adjusting for measured confounders. The pattern of bias was similar when the measured and unmeasured confounders were associated, except that the magnitude of bias was smaller if confounders were positively correlated (details in eAppendix, https://links.lww.com/EDE/A815). Our study shows that the standardized difference can be a useful tool to falsify the third instrumental variable assumption (ie, the instrumental variable is independent of confounders). However, balance of measured confounders between instrumental variable categories does not guarantee balance of unmeasured confounders. If there is an imbalance of measured confounders between instrumental variable categories, indicating a violation of the third assumption, researchers should consider refraining from instrumental variable analysis, irrespective of possible adjustment for measured confounders. M. Sanni Ali Md. Jamal Uddin Division of Pharmacoepidemiology and Clinical Pharmacology Utrecht Institute for Pharmaceutical Sciences University of Utrecht Utrecht, The Netherlands R. H. H. Groenwold Julius Center for Health Sciences and Primary Care University Medical Center Utrecht Utrecht, The Netherlands W. R. Pestman Department of Mathematical Psychology Catholic University of Leuven Leuven, Belgium S. V. Belitser Division of Pharmacoepidemiology and Clinical Pharmacology Utrecht Institute for Pharmaceutical Sciences University of Utrecht Utrecht, The Netherlands A. W. Hoes Julius Center for Health Sciences and Primary Care University Medical Center Utrecht Utrecht, The Netherlands A. de Boer Division of Pharmacoepidemiology and Clinical Pharmacology Utrecht Institute for Pharmaceutical Sciences University of Utrecht Utrecht, The Netherlands K. C. B. Roes Julius Center for Health Sciences and Primary Care University Medical Center Utrecht Utrecht, The Netherlands Olaf H. Klungel Division of Pharmacoepidemiology and Clinical Pharmacology Utrecht Institute for Pharmaceutical Sciences University of Utrecht Utrecht, The Netherlands [email protected]