Quantitative Bias Analysis Research Articles

Data Quality in Electronic Health Record Research: An Approach for Validation and Quantitative Bias Analysis for Imperfectly Ascertained Health Outcomes Via Diagnostic Codes.

It is incumbent upon all researchers who use the electronic health record (EHR), including data scientists, to understand the quality of such data. EHR data may be subject to measurement error or misclassification that have the potential to bias results, unless one applies the available computational techniques specifically created for this problem. In this article, we begin with a discussion of data-quality issues in the EHR focusing on health outcomes. We review the concepts of sensitivity, specificity, positive and negative predictive values, and demonstrate how the imperfect classification of a dichotomous outcome variable can bias an analysis, both in terms of prevalence of the outcome, and relative risk of the outcome under one treatment regime (aka exposure) compared to another. This is then followed by a description of a generalizable approach to probabilistic (quantitative) bias analysis using a combination of regression estimation of the parameters that relate the true and observed data and application of these estimates to adjust the prevalence and relative risk that may have existed if there was no misclassification. We describe bias analysis that accounts for both random and systematic errors and highlight its limitations. We then motivate a case study with the goal of validating the accuracy of a health outcome, chronic infection with hepatitis C virus, derived from a diagnostic code in the EHR. Finally, we demonstrate our approaches on the case study and conclude by summarizing the literature on outcome misclassification and quantitative bias analysis.

A Quantitative Bias Analysis to Assess Constancy Assumption in Noninferiority Trials Using Bayesian Hierarchical Models

Noninferiority (NI) trial designs are often used to compare a new treatment with an active control for treating diseases for which use of a placebo control in a trial is ethically unacceptable. The validity of conclusion from an NI trial relies on some critical assumptions such as constancy assumption. This assumption, often unverifiable in a two-arm NI trial without a placebo control, refers to as the similarity of the active control effect in the current NI trial to that in the historical trials. Violation of the constancy assumption may lead to not only the inflation of Type I error, but also invalid conclusion of the NI trial. In this article, we propose a probabilistic bias analysis of historical data using the Bayesian hierarchical modeling framework to assess heterogeneity of active control effect and hence the constancy assumption. Both random sampling error and heterogeneity in the active control effect across historical trials will be accounted in the analysis of the current NI trial. The proposed method is illustrated using a real example of NI trial with a binary endpoint.

Health technology assessments and real-world evidence: tell us what you want, what you really, really want.

Health technology assessments and real-world evidence: tell us what you want, what you really, really want.

Pregnancy outcomes in women living with HIV and HIV-negative women in South Africa: Cohort analysis based on bias-corrected gestational age.

Antiretroviral therapy (ART) use during pregnancy may be associated with adverse outcomes, but findings have been inconsistent, at least in part due to unreliably estimated gestational age. To quantify the association between HIV status, ART initiation timing and adverse birth outcomes, with reliably assessed gestational age at booking, in a public sector primary care facility in Cape Town, South Africa. Pregnant women, HIV-negative or living with HIV (WLHIV), were enrolled at first antenatal care visit and followed through delivery. Ultrasound-assessed gestational age was deemed the gold standard. Based on quantitative bias analysis for outcome misclassification, gestational age by non-ultrasound assessment was corrected using multiple overimputation, which deals with missing data and measurement error simultaneously. Using bias-corrected gestational age, birth outcomes were compared between WLHIV and HIV-negative women, and among WLHIV who initiated ART before versus during pregnancy, further divided into trimesters. Of 3952 women enrolled, 37% were WLHIV (mostly using tenofovir+emtricitabine+efavirenz). Last menstrual period (LMP)-based gestational age was identified to be biased, and LMP measures were thus corrected using multiple overimputation. Comparing WLHIV and HIV-negative women, adjusted risk ratio (aRR) of overall pregnancy loss was 1.26 (95% confidence interval [CI] 0.98, 1.61); aRR of preterm delivery was 1.02 (95% CI 0.88, 1.20); aRR of small for gestational age infants was 1.43 (95% CI 1.14, 1.80). Among WLHIV, outcomes were similar by ART initiation timing. In this routine care cohort, risk of SGA, and possibly of pregnancy loss, was increased in WLHIV compared with HIV-negative women, with no evidence of increased risk of preterm delivery. Further research is needed to improve mechanistic understanding of the contribution of ART to adverse birth outcomes to optimize treatment for pregnant WLHIV and ensure optimal maternal and infant outcomes.

OP208 Did Health Technology Assessments Make the Wrong Call? Quantitative Bias Analysis: Alectinib versus Ceritinib in Non-Small Cell Lung Cancer

IntroductionThe German health technology assessment (HTA) rejected additional benefit of alectinib for second line (2L) ALK+ NSCLC, citing possible biases from missing ECOG performance status data and unmeasured confounding in real-world evidence (RWE) for 2L ceritinib that was submitted as a comparator to the single arm alectinib trial. Alectinib was approved in the US and therefore US post-launch RWE can be used to evaluate this HTA decision.MethodsWe compared the real-world effectiveness of alectinib with ceritinib in 2L post-crizotinib ALK+ NSCLC using the nationwide Flatiron Health electronic health record (EHR)-derived de-identified database. Using quantitative bias analysis (QBA), we estimated the strength of (i) unmeasured confounding and (ii) deviation from missing-at-random (MAR) assumptions needed to nullify any overall survival (OS) benefit.ResultsAlectinib had significantly longer median OS than ceritinib in complete case analysis. The estimated effect size (Hazard Ratio: 0.55) was robust to risk ratios of unmeasured confounder-outcome and confounder-exposure associations of <2.4.Based on tipping point analysis, missing baseline ECOG performance status for ceritinib-treated patients (49% missing) would need to be more than 3.4-times worse than expected under MAR to nullify the OS benefit observed for alectinib.ConclusionsOnly implausible levels of bias reversed our conclusions. These methods could provide a framework to explore uncertainty and aid decision-making for HTAs to enable patient access to innovative therapies.

Misclassification of Neonatal Abstinence Syndrome Surveillance Estimates: Is Considering the Positive Predictive Value Enough?

Validation studies estimating the positive predictive value (PPV) of neonatal abstinence syndrome (NAS) have consistently suggested overreporting in hospital discharge records. However, few studies estimate the negative predictive value (NPV). Even slightly imperfect NPVs have the potential to bias estimated prevalences of rare outcomes like NAS. Given the challenges in estimating NPV, our objective was to evaluate whether the PPV was sufficient to understand the influence of NAS misclassification bias on conclusions of the NAS prevalence in surveillance research. We used hospital discharge data from the 2016 New Jersey State Inpatient Databases, Healthcare Cost and Utilization Project. We adjusted surveillance data for misclassification using quantitative bias analysis models to estimate the expected NAS prevalence under a range of PPV and NPV bias scenarios. The 2016 observed NAS prevalence was 0.61%. The misclassification-adjusted prevalence estimates ranged from 0.31% to 0.91%. When PPV was assumed to be ≥90%, the misclassification-adjusted prevalence was typically greater than the observed prevalence but the reverse was true for PPV ≤70%. Under PPV 80%, the misclassification-adjusted prevalence was less than the observed prevalence for NPV >99.9% but flipped for NPV <99.9%. When we varied the NPV below 100%, our results suggested that the direction of bias (over or underestimation) was dependent on the PPV, and sometimes dependent on the NPV. However, NPV was important for understanding the magnitude of bias. This study serves as an example of how quantitative bias analysis methods can be applied in NAS surveillance to supplement existing validation data when NPV estimates are unavailable.

Monte Carlo Simulation Approaches for Quantitative Bias Analysis: A Tutorial.

Quantitative bias analysis can be used to empirically assess how far study estimates are from the truth (i.e., an estimate that is free of bias). These methods can be used to explore the potential impact of confounding bias, selection bias (collider stratification bias), and information bias. Quantitative bias analysis includes methods that can be used to check the robustness of study findings to multiple types of bias and methods that use simulation studies to generate data and understand the hypothetical impact of specific types of bias in a simulated data set. In this article, we review 2 strategies for quantitative bias analysis: 1) traditional probabilistic quantitative bias analysis and 2) quantitative bias analysis with generated data. An important difference between the 2 strategies relates to the type of data (real vs. generated data) used in the analysis. Monte Carlo simulations are used in both approaches, but the simulation process is used for different purposes in each. For both approaches, we outline and describe the steps required to carry out the quantitative bias analysis and also present a bias-analysis tutorial demonstrating how both approaches can be applied in the context of an analysis for selection bias. Our goal is to highlight the utility of quantitative bias analysis for practicing epidemiologists and increase the use of these methods in the epidemiologic literature.

Assessment of Alectinib vs Ceritinib in ALK-Positive Non–Small Cell Lung Cancer in Phase 2 Trials and in Real-world Data

Quantitative assessment of bias from unmeasured confounding and missing data can help evaluate uncertainty in findings from indirect comparisons using real-world data (RWD). To compare the effectiveness of alectinib vs ceritinib in terms of overall survival (OS) in patients with ALK-positive, crizotinib-refractory, non-small cell lung cancer (NSCLC) and to assess the sensitivity of these findings to unmeasured confounding and missing data assumptions. This comparative effectiveness research study compared patients from 2 phase 2 alectinib trials and real-world patients. Patients were monitored from June 2013 to March 2020. Comparisons of interest were between alectinib trial data vs ceritinib RWD and alectinib RWD vs ceritinib RWD. RWD treatment groups were selected from nationally representative cancer data from US cancer clinics, the majority from community centers. Participants were ALK-positive patients aged 18 years or older with advanced NSCLC, prior exposure to crizotinib, and Eastern Cooperative Oncology Group Performance Status (PS) of 0 to 2. Data analysis was performed from October 2020 to March 2021. Initiation of alectinib or ceritinib therapy. The main outcome was OS. In total, there were 355 patients: 183 (85 men [46.4%]) in the alectinib trial, 91 (43 men [47.3%]) in the ceritinib RWD group, and 81 (38 men [46.9%]) in the alectinib RWD group. Patients in the alectinib trial were younger (mean [SD] age, 52.53 [11.18] vs 57.97 [11.71] years), more heavily pretreated (mean [SD] number of prior therapy lines, 1.95 [0.72] vs 1.47 [0.81]), and had more favorable baseline ECOG PS (ECOG PS of 0 or 1, 165 patients [90.2%] vs 37 patients [77.1%]) than those in the ceritinib RWD group. The alectinib RWD group (mean [SD] age, 58.69 [11.26] years) had more patients with favorable ECOG PS (ECOG PS of 0 or 1, 49 patients [92.4%] vs 37 patients [77.1%]) and more White patients (56 patients [72.7%] vs 53 patients [62.4%]) compared with the ceritinib group. Compared with ceritinib RWD, alectinib-exposed patients had significantly longer OS in alectinib trials (adjusted hazard ratio [HR], 0.59; 95% CI, 0.44-0.75; P < .001) and alectinib RWD (HR, 0.46; 95% CI, 0.29-0.63; P < .001) after adjustment for baseline confounders. For the worst-case HR estimate of 0.59, residual confounding by a hypothetical confounder associated with mortality and treatment by a risk ratio greater than 2.24 was required to reverse the findings. Conclusions were robust to plausible deviations from random missingness for missing ECOG PS and underrecorded comorbidities and central nervous system metastases in RWD. Alectinib exposure was associated with longer OS compared with ceritinib in patients with ALK-positive NSCLC, and only substantial levels of bias examined reversed the findings. These findings suggest that quantitative bias analysis can be a useful tool to address uncertainty of findings for decision-makers considering RWD.

Mortality following workplace injury: Quantitative bias analysis

Recent studies have shown increased all-cause mortality among workers following disabling workplace injury. These studies did not account for 2 potentially important confounders, smoking and obesity. We estimated injury-related mortality accounting for these factors. We followed workers receiving New Mexico workers' compensation benefits (1994-2000) through 2013. Using data from the Panel Study of Income Dynamics, we derived the joint distribution of smoking status and obesity for workers with and without lost-time injuries. We conducted a quantitative bias analysis (QBA) to determine the adjusted relationship of injury and mortality. We observed hazard ratios after adjusting for smoking and obesity of 1.13 for women (95% simulation interval (SI) 0.97 to 1.31) and 1.12 for men (95% SI 1.00 to 1.27). The estimated fully adjusted excess hazard was about half the estimates not adjusted for these factors. Using QBA to adjust for smoking and obesity reduced the estimated mortality hazard from lost-time injuries and widened the simulation interval. The adjusted estimate still showed more than a 10 percent increase for both women and men. The change in estimates reveals the importance of accounting for these confounders. Of course, the results depend on the methods and assumptions used.

Invited Commentary: Intermittent Opioid Use and Ischemic Placental Disease-Clarifying Associations With Adverse Pregnancy Outcomes.

Discomfort and, to a lesser extent, pain are common complaints during pregnancy, and some patients may turn to opioids for pain relief. Esposito et al. (Am J Epidemiol. 2022;191(5):759-768) report associations between intermittent exposure to opioids during pregnancy and the risk of ischemic placental disease-a syndrome that includes preeclampsia, placental abruption, births that are small for gestational age, and preterm delivery. They found that early opioid exposure in pregnancy was associated with a modestly increased risk for abruption, births that are small for gestational age, and preterm delivery, and both early and late exposures were associated with the greatest risk for these outcomes. Surprisingly, preeclampsia was not associated with opioid use. Through quantitative bias analysis, the authors cleverly tackle a number of biases to assess their roles in explaining the associations, including unmeasured confounding, outcome misclassification, and residual confounding; none exerted strong influences on the associations. Although the findings appear fairly robust on the surface, the lack of association between intermittent opioid use and preeclampsia, and important differences in characteristics of patients in the opioid-exposed group compared with the unexposed group, suggest that further study is needed to clarify the relationship between intermittent opioid use, lifestyle factors, and ischemic placental disease risk.

A quantitative analysis of accuracy, reliability and bias in judgements of functional analyses.

Functional analysis can be considered a diagnostic assessment that behavior analysts use to determine behavioral function. Such a diagnosis ultimately requires a yes or no decision (i.e., a variable maintains a behavior, or it does not) that is determined by both subjective (clinical judgement) and objective (data) variables. Accurate and reliable identification of function is essential for successful treatment, yet behavior analysts' interpretation of data relies on their ability to detect visual differences in graphed data. Some research indicates that behavior analysts have questionable reliability in their visual analysis. To further examine the reliability, accuracy, and bias in visual analysis of functional analysis graphs, we simulated functional analysis results and surveyed 121 BCBA-Ds experienced in visual analysis. We then examined reliability of responses and used a signal detection theory approach to analyze accuracy and bias. Findings suggest that reliability and accuracy of judgements are questionable, and exploration of decision aids is warranted.

Quantitative Bias Analysis of the Association between Occupational Radiation Exposure and Ischemic Heart Disease Mortality in UK Nuclear Workers.

The scientific question of whether protracted low-dose or low-dose-rate exposure to external radiation is causally related to the risk of circulatory disease continues to be an important issue for radiation protection. Previous analyses of a matched case-control dataset nested in a large cohort of UK nuclear fuel cycle workers indicated that there was little evidence that observed associations between external radiation dose and ischemic heart disease (IHD) mortality risk [OR = 1.35 (95% CI: 0.99-1.84) for 15-year-lagged exposure] could alternatively be explained by confounding from pre-employment tobacco smoking, BMI or blood pressure, or from socioeconomic status or occupational exposure to excessive noise or shiftwork. To improve causal inference about the observed external radiation dose and IHD mortality association, we estimated the potential magnitude and direction of non-random errors, incorporated sensitivity analyses and simulated bias effects under plausible scenarios. We conducted quantitative bias analyses of plausible scenarios based on 1,000 Monte Carlo samples to explore the impact of exposure measurement error, missing information on tobacco smoking, and unmeasured confounding, and assessed whether observed associations were reliant on the inclusion of specific matched pairs using bootstrapping with 10% of matched pairs randomly excluded in 1,000 samples. We further explored the plausibility that having been monitored for internal exposure, which was an important confounding factor in the case-control analysis for which models were adjusted, was indeed a confounding factor or whether it might have been the result of some form of selection bias. Consistent with the broader epidemiological evidence-base, these analyses provide further evidence that the dose-response association between cumulative external radiation exposure and IHD mortality is non-linear in that it has a linear shape plateauing at an excess risk of 43% (95% CI: 7-92%) on reaching 390 mSv. Analyses of plausible scenarios of patterns of missing data for tobacco smoking at start of employment indicated that this resulted in relatively little bias towards the null in the original analysis. An unmeasured confounder would have had to have been highly correlated (rp > 0.60) with cumulative external radiation dose to importantly bias observed associations. The confounding effect of "having been monitored for internal dose" was unlikely to have been a true confounder in a biological sense, but instead may have been some unknown factor related to differences over time and between sites in selection criteria for internal monitoring, possibly resulting in collider bias. Plausible patterns of exposure measurement error negatively biased associations regardless of the modeled scenario, but did not importantly change the shape of the observed dose-response associations. These analyses provide additional support for the hypothesis that the observed association between external radiation exposure and IHD mortality may be causal.

Use of register- and survey-based measures of anxiety in a population-based Danish cohort.

We explored the comparability of anxiety measures from register- and survey-based data including analyses of prevalence and associations with selected psychiatric and somatic diseases. We measured anxiety using Danish registers (hospital diagnosis and anxiolytic drug prescriptions), self-reports, symptom checklist (SCL) scores, and a clinical interview in 7493 adults with mean age 52 (SD 13.3) years who participated in a health survey between 2012 and 2015. We estimated the prevalence of anxiety, agreement between different measures and performed quantitative bias analysis. The lifetime prevalence of hospital diagnosed anxiety, anxiolytic drug prescriptions, and self-reported anxiety were 4.4%, 6.2%, and 5.1%, respectively, after adjusting for selective participation. The agreement between the different anxiety measures was low. Thus, 25% with an anxiety diagnosis and 20% with anxiolytic drug prescriptions also had a high SCL score. Anxiolytic drugs were the only measure significantly associated with higher odds of heart disease. Hospital diagnosis and self-reported anxiety were associated with depression with odds ratio (OR) above 15, whereas anxiolytic drug prescriptions were less strongly associated (OR=2.2(95% confidence interval: 1.26-3.91)). The risk estimates attenuated considerably when correcting for measurement error, whereas the ORs became slightly higher when the selective participation in the survey was accounted for. Anxiety diagnosed in hospitals and self-reported anxiety showed low level of agreement but provide comparable results regarding frequency measures and associations with disease outcomes.

Cardiac Complications in Pregnancy: A Systematic Review and Meta-Analysis of Diagnostic Accuracy of BNP and N-Terminal Pro-BNP.

IntroductionSerum brain-type natriuretic peptide (BNP) levels may be involved in detecting the breakdown of defense mechanisms in pregnant women. BNP and N-terminal pro-BNP (NT-proBNP) levels can be used in emergency settings as a biomarker to rule out or confirm cardiac complications. The present study was conducted as an attempt to evaluate the performance of BNP and NT-proBNP as diagnostic tools for cardiac complications, including heart failure and pre-eclampsia, in pregnant and recently-delivered women.MethodsThis meta-analysis was performed according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. To extract the data, articles meeting the inclusion criteria were directly reviewed by two of the authors. Disputes were resolved through discussion with all authors before data mining. In cases of unresolved disagreement, a third reviewer was consulted to achieve consensus. A quantitative analysis of the total bias of publications was performed using Stata software version 14.0, using funnel plots and Deek’s test.ResultsOverall, 13 unique studies were included in this review. The pooled sensitivity, specificity and positive and negative predictive values for diagnosing cardiac complications in pregnant women when BNP level was ≤ 100 ng/L, were determined to be 95, 62, 71 and 91%, respectively. The corresponding measures when the BNP levels were 100–500 ng/L were calculated to be 98, 92, 97 and 92%, respectively. The pooled sensitivity, specificity and positive and negative predictive values for diagnosing cardiac complications in pregnant women using NT-proBNP were 78, 74, 20.5 and 97%, respectively.ConclusionOur results suggest that both BNP and NT-proBNP levels can be used as diagnostic tools among pregnant and newly-delivered women to diagnose cardiac complications, including heart failure and pre-eclampsia. While BNP showed a relatively better diagnostic accuracy compared to NT-proBNP, it must be noted that the number of studies evaluating NT-proBNP included in this meta-analysis was low and the studies were inconsistent in terms of cutoff value. Further studies are required to confirm the prospective use of BNP and NT-pro BNP in assessing common symptoms that indicate cardiac complications during labor. Moreover, the clinical use of the NT-proBNP test in this field requires further study.

Assessing Techniques for Quantifying the Impact of Bias Due to an Unmeasured Confounder: An Applied Example

PurposeTo compare the magnitude of bias due to unmeasured confounding estimated from various techniques in an applied example.Patients and MethodsWe examined the association between dibutyl phthalate (DBP) and incident estrogen receptor (ER)-positive breast cancer in a Danish nationwide cohort (N=1,122,042). Cox regression analyses were adjusted for age and active drug compounds contributing to DBP exposure. We estimated the hazard ratios (HRs) that would have been observed had one of the DBP sources been unmeasured and calculated the strength of confounding by comparing to the fully adjusted HR. We performed a quantitative bias analysis (QBA) of the “unmeasured” confounder, using external information to specify the bias parameters. Upper bounds on the bias were estimated and E-values were calculated.ResultsThe adjusted HR for incident ER-positive breast cancer among women with high-level (≥10,000 cumulative milligrams) versus no DBP exposure was 2.12 (95% confidence interval 1.12 to 4.05). Removing each DBP source in isolation resulted in negligible change in the HR. The bias estimates from the QBA ranged from 1.00 to 1.01. The estimated maximum impact of unmeasured confounding ranged from 1.01 to 1.51. E-values ranged from 3.46 to 3.68.ConclusionThe impact of bias due to simulated unmeasured confounding was negligible, in part, because the unmeasured variable was not independent of controlled variables. When a suspected confounder cannot be measured in the study data, our exercise suggests that QBA is the most informative method for assessing the impact. E-values may best be reserved for situations where uncontrolled confounding emanates from an unknown confounder.

The Importance of Making Assumptions in Bias Analysis.

Quantitative bias analyses allow researchers to adjust for uncontrolled confounding, given specification of certain bias parameters. When researchers are concerned about unknown confounders, plausible values for these bias parameters will be difficult to specify. Ding and VanderWeele developed bounding factor and E-value approaches that require the user to specify only some of the bias parameters. We describe the mathematical meaning of bounding factors and E-values and the plausibility of these methods in an applied context. We encourage researchers to pay particular attention to the assumption made, when using E-values, that the prevalence of the uncontrolled confounder among the exposed is 100% (or, equivalently, the prevalence of the exposure among those without the confounder is 0%). We contrast methods that attempt to bound biases or effects and alternative approaches such as quantitative bias analysis. We provide an example where failure to make this distinction led to erroneous statements. If the primary concern in an analysis is with known but unmeasured potential confounders, then E-values are not needed and may be misleading. In cases where the concern is with unknown confounders, the E-value assumption of an extreme possible prevalence of the confounder limits its practical utility.

Multiple-bias Sensitivity Analysis Using Bounds.

Confounding, selection bias, and measurement error are well-known sources of bias in epidemiologic research. Methods for assessing these biases have their own limitations. Many quantitative sensitivity analysis approaches consider each type of bias individually, although more complex approaches are harder to implement or require numerous assumptions. By failing to consider multiple biases at once, researchers can underestimate-or overestimate-their joint impact. We show that it is possible to bound the total composite bias owing to these three sources and to use that bound to assess the sensitivity of a risk ratio to any combination of these biases. We derive bounds for the total composite bias under a variety of scenarios, providing researchers with tools to assess their total potential impact. We apply this technique to a study where unmeasured confounding and selection bias are both concerns and to another study in which possible differential exposure misclassification and confounding are concerns. The approach we describe, though conservative, is easier to implement and makes simpler assumptions than quantitative bias analysis. We provide R functions to aid implementation.

RW3 Quantitative Bias Analysis (QBA) for Comparative Effectiveness of Alectinib versus Ceritinib in Non-Small Cell Lung Cancer (NSCLC)

German and Canadian HTAs rejected reimbursement of alectinib for second-line (2L) ALK+ NSCLC citing possible biases from (i) unmeasured confounding and (ii) missing ECOG performance status (PS) data in real-world evidence (RWE) for 2L ceritinib submitted as comparator to single-arm alectinib trials. We sought to evaluate this HTA decision using post-launch real-world data from the United States, where alectinib was approved. Quantitative bias analysis was used to compare the effectiveness of alectinib versus ceritinib in 2L post-crizotinib ALK+ NSCLC using treatment arms derived from pooled single-arm alectinib trials (n=183) and records from a US nationwide Flatiron Health electronic health record (EHR)-derived de-identified database (n=87 and 102 for alectinib and ceritinib). Propensity score weighting was used to balance baseline characteristics between treatment arms. Large imbalances were observed in age, CNS metastases and prior chemotherapy. After adjustment for baseline covariates, alectinib-treated patients (n=55) had significantly longer median overall survival (OS) of 28 months (95% CI: 20-37) than ceritinib (11 months, 95% CI: 8-27; n=44) using complete case analysis (hazard ratio (HR): 0.64, 0.43-0.91). An unmeasured confounder associated with OS and alectinib exposure by a risk ratio >2.1-fold was needed to nullify the observed treatment effect. Under a missing-at-random assumption (MAR) for ECOG PS scores (49% missing for ceritinib arm) using multiple imputation, the pooled HR was 0.64 (0.5-0.8). The observed treatment effect was robust to all deviations from MAR where balance in baseline characteristics could be achieved. Alectinib maintained a significantly longer median OS (28 months versus 13 months, HR: 0.70, 0.54-0.85) after shifting an additional 10-40% patients to a poorer ECOG PS than expected under MAR. Only implausible levels of bias reversed our conclusions. These methods could provide a framework to explore uncertainty in RWE and aid decision-making for HTAs to enable patient access to innovative therapies.

Sensitivity analyses of unmeasured and partially-measured confounders using multiple imputation in a vaccine safety study.

Sensitivity analyses have played an important role in pharmacoepidemiology studies using electronic health records data. Despite the existence of quantitative bias analysis in pharmacoepidemiologic studies, simultaneously adjusting for unmeasured and partially measured confounders is challenging in vaccine safety studies. Our objective was to develop a flexible approach for conducting sensitivity analyses of unmeasured and partially-measured confounders concurrently for a vaccine safety study. We derived conditional probabilities for an unmeasured confounder based on bias parameters, used these conditional probabilities and Monte Carlo simulations to impute the unmeasured confounder, and re-constructed the analytic datasets as if the unmeasured confounder had been observed. We simultaneously imputed a partially measured confounder using a prediction model. We considered unmeasured breastfeeding and partially measured family history of Type 1 diabetes (T1DM) in a study examining the association between exposure to rotavirus vaccination and T1DM. Before sensitivity analyses, the hazard ratios (HR) were 1.50 (95% CI, 0.81-2.77) for those partially exposed and 1.03 (95% CI, 0.62-1.72) for those fully exposed with unexposed children as the referent group. When breastfeeding and family history of T1DM were adjusted, the HR was 1.55 (95% CI, 0.84-2.87) for the partially exposed group; the HR was 0.98 (95% CI, 0.58-1.63) for the fully exposed group. We conclude that adjusting for unmeasured breastfeeding and partially measured family history of T1DM did not alter the conclusion that there was no evidence of association between rotavirus vaccination and developing T1DM. This novel approach allows for simultaneous adjustment for multiple unmeasured and partially-measured confounders.

Spotlight commentary: A role for real-world evidence to inform the clinical care of patients with diabetes mellitus.

Spotlight commentary: A role for real-world evidence to inform the clinical care of patients with diabetes mellitus.