Balance Of Baseline Covariates Research Articles

Emergency Department Referral of Patients With Chest Pain for Noninvasive Cardiac Testing and 2-Year Clinical Outcomes.

Noninvasive cardiac testing (NICT) has been associated with decreased long-term risks of major adverse cardiac events (MACEs) among emergency department patients at high coronary risk. It is unclear whether this association extends to patients without evidence of myocardial injury on initial ECG and cardiac troponin testing. A retrospective cohort study was conducted of patients presenting with chest pain between 2013 and 2019 to 21 emergency departments within an integrated health care system in Northern California, excluding patients with ST-segment-elevation myocardial infarction or myocardial injury by serum troponin testing. To account for confounding by indication, we grouped patient encounters by the NICT referral rate of the initially assigned emergency physician relative to local peers within discrete time periods. The primary outcome was MACE within 2 years. Secondary outcomes were coronary revascularization and MACE, inclusive of all-cause mortality. Associations between the NICT referral group (low, intermediate, or high) and outcomes were assessed using risk-adjusted proportional hazards methods with censoring for competing events. Among 144 577 eligible patient encounters, the median age was 58 years (interquartile range, 48-68) and 57% were female. Thirty-day NICT referral was 13.0%, 19.9%, and 27.8% in low, intermediate, and high NICT referral groups, respectively, with a good balance of baseline covariates between groups. Compared with the low NICT referral group, there was no significant decrease in the adjusted hazard ratio of MACE within the intermediate (adjusted hazard ratio, 1.08 [95% CI, 1.02-1.14]) or high (adjusted hazard ratio, 1.05 [95% CI, 0.99-1.11]) NICT referral groups. Results were similar for MACE, inclusive of all-cause mortality, and coronary revascularization, as well as subgroup analyses stratified by estimated risk (history, electrocardiogram, age, risk factors, troponin [HEART] score: percent classified as low risk, 48.2%; moderate risk, 49.2%; and high risk, 2.7%). Increases in NICT referrals were not associated with changes in the hazard of MACE within 2 years following emergency department visits for chest pain without evidence of acute myocardial injury. These findings further highlight the need for evidence-based guidance regarding the appropriate use of NICT in this population.

Autologous cell salvage in off-pump coronary artery bypass surgery reduces post-operative complications: a retrospective weighted-matching analysis.

Blood transfusion plays a crucial role in coronary artery bypass grafting (CABG). The choice between autologouscell saver (CS) and allogenic blood transfusion (ABT) has been a continuous debate in the medical community, especially within cardiac surgery. This study aimed to assess the outcomes of off-pump CABG (OPCAB) surgery in patients receiving blood solely via cell salvage compared to those receiving ABT or a combination of ABT and CS perioperatively. A total of 414 patients who underwent isolated OPCAB surgery at our cardiovascular clinic were analyzed. Among them, 250 patients (60.4%) received blood via CS alone, while 164 patients (39.6%) received either ABT or a mix of ABT and CS. Stabilized inverse probability treatment weighted (IPTW) matching technique ensured balance in baseline covariates. We found no significant differences in 30-day mortality rates between the CS and ABT groups. The CS group displayed significantly lower rates of overall complications, encompassing stroke, acute kidney injury, atrial fibrillation, and pulmonary complications. Rates of sepsis, readmission, gastrointestinal complications, heparin-induced thrombosis, and deep venous thrombosis were comparable between the two groups. However, in contrast to the ABT group,the CS group exhibited significantly shorter median lengths of hospital stay (LOHS), ICU stay, andventilation time, along with higher rates of discharge to homerather than acute care facilities. Our data suggest that autologous blood transfusion via CS results in fewer perioperative complications and faster recovery following OPCAB procedures as compared to ABT.

An evolutionary algorithm for the direct optimization of covariate balance between nonrandomized populations.

Matching reduces confounding bias in comparing the outcomes of nonrandomized patient populations by removing systematic differences between them. Under very basic assumptions, propensity score (PS) matching can be shown to eliminate bias entirely in estimating the average treatment effect on the treated. In practice, misspecification of the PS model leads to deviations from theory and matching quality is ultimately judged by the observed post-matching balance in baseline covariates. Since covariate balance is the ultimate arbiter of successful matching, we argue for an approach to matching in which the success criterion is explicitly specified and describe an evolutionary algorithm to directly optimize an arbitrary metric of covariate balance. We demonstrate the performance of the proposed method using a simulated dataset of 275,000 patients and 10 matching covariates. We further apply the method to match 250 patients from a recently completed clinical trial to a pool of more than 160,000 patients identified from electronic health records on 101 covariates. In all cases, we find that the proposed method outperforms PS matching as measured by the specified balance criterion. We additionally find that the evolutionary approach can perform comparably to another popular direct optimization technique based on linear integer programming, while having the additional advantage of supporting arbitrary balance metrics. We demonstrate how the chosen balance metric impacts the statistical properties of the resulting matched populations, emphasizing the potential impact of using nonlinear balance functions in constructing an external control arm. We release our implementation of the considered algorithms in Python.

Minimization in randomized clinical trials.

In randomized trials, comparability of the treatment groups is ensured through allocation of treatments using a mechanism that involves some random element, thus controlling for confounding of the treatment effect. Completely random allocation ensures comparability between the treatment groups for all known and unknown prognostic factors. For a specific trial, however, imbalances in prognostic factors among the treatment groups may occur. Although accidental bias can be avoided in the presence of such imbalances by stratifying the analysis, most trialists, regulatory agencies, and other stakeholders prefer a balanced distribution of prognostic factors across the treatment groups. Some procedures attempt to achieve balance in baseline covariates, by stratifying the allocation for these covariates, or by dynamically adapting the allocation using covariate information during the trial (covariate-adaptive procedures). In this Tutorial, the performance of minimization, a popular covariate-adaptive procedure, is compared with two other commonly used procedures, completely random allocation and stratified blocked designs. Using individual patient data of 2 clinical trials (in advanced ovarian cancer and age-related macular degeneration), the procedures are compared in terms of operating characteristics (using asymptotic and randomization tests), predictability of treatment allocation, and achieved balance. Fifty actual trials of various sizes that applied minimization for treatment allocation are used to investigate the achieved balance. Implementation issues of minimization are described. Minimization procedures are useful in all trials but especially when (1) many major prognostic factors are known, (2) many centers of different sizes accrue patients, or (3) the trial sample size is moderate.

A randomization-based theory for preliminary testing of covariate balance in controlled trials

Randomized trials balance all covariates on average and are the gold standard for estimating treatment effects. Chance imbalances nevertheless exist more or less in realized treatment allocations and intrigue an important question: what should we do if the treatment groups differ with respect to some important baseline characteristics? A common strategy is to conduct a preliminary test of the balance of baseline covariates after randomization, and invoke covariate adjustment for subsequent inference if and only if the realized allocation fails some prespecified criterion. Although such practice is intuitive and popular among practitioners, the existing literature has so far only evaluated its properties under strong parametric model assumptions in theory and simulation, yielding results of limited generality. To fill this gap, we examine two strategies for conducting preliminary test-based covariate adjustment by regression, and evaluate the validity and efficiency of the resulting inferences from the randomization-based perspective. The main result is twofold. First, the preliminary-test estimator based on the analysis of covariance can be even less efficient than the unadjusted difference in means, and risks anticonservative confidence intervals based on normal approximation even with the robust standard error. Second, the preliminary-test estimator based on the fully interacted specification is less efficient than its counterpart under the always-adjust strategy, and yields overconservative confidence intervals based on normal approximation. In addition, although the Fisher randomization test is still finite-sample exact for testing the sharp null hypothesis of no treatment effect on any individual, it is no longer valid for testing the weak null hypothesis of zero average treatment effect in large samples even with properly studentized test statistics. These undesirable properties are due to the asymptotic non-normality of the preliminary-test estimators. Based on theory and simulation, we echo the existing literature and do not recommend the preliminary-test procedure for covariate adjustment in randomized trials.

Clinical outcome after total neoadjuvant treatment (CAO/ARO/AIO-12) versus intensified neoadjuvant and adjuvant treatment (CAO/ARO/AIO-04) a comparison between two multicenter randomized phase II/III trials

BackgroundTotal neoadjuvant therapy (TNT) can enhance local tumor regression, but its survival benefits compared to intensified chemoradiotherapy (CRT) followed by adjuvant chemotherapy (CT) remain unclear. MethodsThis is a secondary comparison between 607 patients treated with intensified 5-FU/Oxaliplatin neoadjuvant CRT and adjuvant CT within the experimental arm of the CAO/ARO/AIO-04 phase III trial, and 306 patients treated with TNT within the CAO/ARO/AIO-12 phase II trial. Comparison between clinical-pathological characteristics, surgical quality, and post-surgical complications were analyzed using the Pearson’s Chi-squared or Mann-Whitney U test. Oncological outcome was examined with log-rank, Gray’s test, and multivariate cox regression. In addition, further subgroup analyses and propensity score matching were performed to optimize the balance of baseline covariates. FindingsPatients treated with CRT followed by consolidation CT had a significantly higher rate of pathological complete remission (pCR) compared to patients treated within the experimental arm of the CAO/ARO/AIO-04 trial (25.3 % vs 17.3 %, P = 0.04). Post-surgical complications were less common in the CAO/ARO/AIO-12 trial. After a median follow-up of 46 months, clinical outcome did not differ significantly in the overall cohort, in any subgroup or after propensity score matching. In multivariate analysis, disease-free survival (DFS) was similar between the experimental arm of the CAO/ARO/AIO-04 trial and treatments arms of the CAO/ARO/AIO-12 trial (vs arm A: HR 0.92 [95 % CI 0.62–1.37], P = 0.69; vs arm B: HR 1.06 [95 % CI 0.72–1.58], P = 0.76). InterpretationNotwithstanding the limitations of intertrial comparison, TNT did not improve long term oncological outcome in our study compared to the intensified neoadjuvant CRT and adjuvant CT treatment in the CAO/ARO/AIO-04 trial. Improved response rates after TNT offers an attractive option to explore organ preservation in selective patients with locally advanced rectal cancer.

Performance of LTMLE in the Presence of Missing Data in Control-Matched Longitudinal Studies

Conventional controlled trials employ randomization and blinding to ensure the balance of baseline covariates between study arms. Treatment effect is formally tested via a pre-specified hypothesis reflecting trial’s primary objective defined by the primary efficacy endpoint, such as, is experimental treatment effective in slowing cognitive decline at a pre-specified landmark time in a neurologic therapeutic development? To address the same clinical question, but, as a safety endpoint in an observational study post radiopharmaceutical imaging drug approval due to concerns of radiation retention in the brain after accumulated real-world use over time, the targeted minimum loss-based estimation (TMLE) method has been suggested. Overall, for a longitudinal control-matched cohort study, assessing treatment effect (efficacy or safety) in the presence of missingness can be very challenging, which depends also on trial duration and missingness pattern of outcome data between study arms. TMLE is a two-step procedure to estimate a target parameter and has been shown to be doubly robust. The objectives of our research are a few folds. First, we investigate the performance of the TMLE method using longitudinal structure (LTMLE) in a prospective control-matched longitudinal cohort study. Second, we evaluate the performance of LTMLE and a few methods that are often proposed in regulatory submissions for longitudinal data analysis or in the presence of missing data with various missing data mechanisms. Third, we assess the impacts of various missing data mechanisms to treatment effect estimation via extensive simulation studies. Finally, we discuss the results of the simulation studies and their implications to conducting a feasible real-world control-matched longitudinal cohort study.

Evaluation of a geriatrics primary care model using prospective matching to guide enrollment

BackgroundFew definitive guidelines exist for rigorous large-scale prospective evaluation of nonrandomized programs and policies that require longitudinal primary data collection. In Veterans Affairs (VA) we identified a need to understand the impact of a geriatrics primary care model (referred to as GeriPACT); however, randomization of patients to GeriPACT vs. a traditional PACT was not feasible because GeriPACT has been rolled out nationally, and the decision to transition from PACT to GeriPACT is made jointly by a patient and provider. We describe our study design used to evaluate the comparative effectiveness of GeriPACT compared to a traditional primary care model (referred to as PACT) on patient experience and quality of care metrics.MethodsWe used prospective matching to guide enrollment of GeriPACT-PACT patient dyads across 57 VA Medical Centers. First, we identified matches based an array of administratively derived characteristics using a combination of coarsened exact and distance function matching on 11 identified key variables that may function as confounders. Once a GeriPACT patient was enrolled, matched PACT patients were then contacted for recruitment using pre-assigned priority categories based on the distance function; if eligible and consented, patients were enrolled and followed with telephone surveys for 18 months.ResultsWe successfully enrolled 275 matched dyads in near real-time, with a median time of 7 days between enrolling a GeriPACT patient and a closely matched PACT patient. Standardized mean differences of < 0.2 among nearly all baseline variables indicates excellent baseline covariate balance. Exceptional balance on survey-collected baseline covariates not available at the time of matching suggests our procedure successfully controlled many known, but administratively unobserved, drivers of entrance to GeriPACT.ConclusionsWe present an important process to prospectively evaluate the effects of different treatments when randomization is infeasible and provide guidance to researchers who may be interested in implementing a similar approach. Rich matching variables from the pre-treatment period that reflect treatment assignment mechanisms create a high quality comparison group from which to recruit. This design harnesses the power of national administrative data coupled with collection of patient reported outcomes, enabling rigorous evaluation of non-randomized programs or policies.

Propensity Score Analysis with Partially Observed Baseline Covariates: A Practical Comparison of Methods for Handling Missing Data

(1) Background: Propensity score methods gained popularity in non-interventional clinical studies. As it may often occur in observational datasets, some values in baseline covariates are missing for some patients. The present study aims to compare the performances of popular statistical methods to deal with missing data in propensity score analysis. (2) Methods: Methods that account for missing data during the estimation process and methods based on the imputation of missing values, such as multiple imputations, were considered. The methods were applied on the dataset of an ongoing prospective registry for the treatment of unprotected left main coronary artery disease. The performances were assessed in terms of the overall balance of baseline covariates. (3) Results: Methods that explicitly deal with missing data were superior to classical complete case analysis. The best balance was observed when propensity scores were estimated with a method that accounts for missing data using a stochastic approximation of the expectation-maximization algorithm. (4) Conclusions: If missing at random mechanism is plausible, methods that use missing data to estimate propensity score or impute them should be preferred. Sensitivity analyses are encouraged to evaluate the implications methods used to handle missing data and estimate propensity score.

Hemodynamic Response to Vasopressin Dosage of 0.03 Units/Min vs. 0.04 Units/Min in Patients With Septic Shock.

Arginine vasopressin (AVP) is suggested as an adjunct to norepinephrine in patients with septic shock. Guidelines recommend an AVP dosage up to 0.03 units/min, but 0.04 units/min is commonly used in practice based on initial studies. This study was designed to compare the incidence of hemodynamic response between initial fixed-dosage AVP 0.03 units/min and AVP 0.04 units/min. This retrospective, multi-hospital health system, cohort study included adult patients with septic shock receiving AVP as an adjunct to catecholamine vasopressors. Patients were excluded if they received an initial dosage other than 0.03 units/min or 0.04 units/min, or AVP was titrated within the first 6 hours of therapy. The primary outcome was hemodynamic response, defined as a mean arterial pressure ≥65 mm Hg and a decrease in catecholamine dosage at 6 hours after AVP initiation. Inverse probability of treatment weighting (IPTW) based on the propensity score for initial AVP dosage receipt was utilized to estimate adjusted exposure effects. Of the 1536 patients included in the observed data, there was a nearly even split between initial AVP dosage of 0.03 units/min (n = 842 [54.8%]) and 0.04 units/min (n = 694 [45.2%]). Observed patients receiving AVP 0.03 units/min were more frequently treated at the main campus academic medical center (96.3% vs. 52.2%, p < 0.01) and in a medical intensive care unit (87.4% vs. 39.8%, p < 0.01). The IPTW analysis included 1379 patients with achievement of baseline covariate balance. There was no evidence for a difference between groups in the incidence of hemodynamic response (0.03 units/min 50.0% vs. 0.04 units/min 53.1%, adjusted relative risk 1.06 [95% CI 0.94, 1.20]). Initial AVP dosing varied by hospital and unit type. Although commonly used, an initial AVP dosage of 0.04 units/min was not associated with a higher incidence of early hemodynamic response to AVP in patients with septic shock.

Evaluation of a Package of Behaviour Change Interventions (Baduta Program) to Improve Maternal and Child Nutrition in East Java, Indonesia: Protocol for an Impact Study.

BackgroundOver the past decade, the prevalence of stunting has been close to 37% in children aged <5 years in Indonesia. The Baduta program, a multicomponent package of interventions developed by the Global Alliance for Improved Nutrition, aims to improve maternal and infant nutrition in Indonesia.ObjectiveThis study aims to assess the impact of the Baduta program, a package of health system strengthening and behavior change interventions, compared with the standard village health services on maternal and child nutrition.MethodsThe impact evaluation uses a cluster randomized controlled trial design with 2 outcome assessments. The first uses cross-sectional surveys of mothers of children aged 0-23 months and pregnant women before and after the interventions. The second is a cohort study of pregnant women followed until their child is 18 months from a subset of clusters. We will also conduct a process evaluation guided by the program impact pathway to assess coverage, fidelity, and acceptance. The study will be conducted in the Malang and Sidoarjo districts of East Java, Indonesia. The unit of randomization is the subdistricts. As random allocation of interventions to only 6 subdistricts is feasible, we will use constrained randomization to ensure balance of baseline covariates. The first intervention will be health system strengthening, including the Baby-Friendly Hospital Initiative, and training on counseling for appropriate infant and young child feeding (IYCF). The second intervention will be nutrition behavior change that includes Emo-Demos; a national television (TV) advertising campaign; local screening TV spots; a free, text message service; and promotion of low-cost water filters and hygiene practices. The primary study outcome is child stunting (low length-for-age), and secondary outcomes include length-for-age Z scores, wasting (low weight-for-length), anemia, child morbidity, IYCF indicators, and maternal and child nutrient intakes. The sample size for each cross-sectional survey is 1400 mothers and their children aged <2 years and 200 pregnant women in each treatment group. The cohort evaluation requires a sample size of 340 mother-infant pairs in each treatment group. We will seek Gatekeeper consent and written informed consent from the participants. The intention-to-treat principle will guide our data analysis, and we will apply Consolidated Standards of Reporting Trials guidelines for clustered randomized trials in the analysis.ResultsIn February 2015, we conducted a baseline cross-sectional survey on 2435 women with children aged <2 years and 409 pregnant women. In February 2017, we conducted an end-line survey on 2740 mothers with children aged <2 years and 642 pregnant women. The cohort evaluation began in February 2015, with 729 pregnant women, and was completed in December 2016.ConclusionsThe results of the program evaluation will help guide policies to support effective packages of behavior change interventions to prevent child stunting in Indonesia.International Registered Report Identifier (IRRID)RR1-10.2196/18521

An omnibus approach to assess covariate balance in observational studies using the distance covariance.

Adequate baseline covariate balance among groups is critical in observational studies designed to estimate causal effects. Propensity score-based methods are popular ways to achieve covariate balance among groups. Existing methods are not easily generalizable to situations in which covariates of mixed type are collected nor do they provide a convenient way to compare the overall covariate vector distributions. Instead, covariate balance is assessed at the individual covariate level, thus the potential for increased overall type I error. We propose the use of the distance covariance, developed by Székely and colleagues, as an omnibus test of independence between covariate vectors and study group. We illustrate the advantages of this methodology in simulated data and in a cardiac surgery study designed to assess the impact of preoperative statin therapy on outcomes.

Assessing covariate balance when using the generalized propensity score with quantitative or continuous exposures.

Propensity score methods are increasingly being used to estimate the effects of treatments and exposures when using observational data. The propensity score was initially developed for use with binary exposures (e.g., active treatment vs. control). The generalized propensity score is an extension of the propensity score for use with quantitative exposures (e.g., dose or quantity of medication, income, years of education). A crucial component of any propensity score analysis is that of balance assessment. This entails assessing the degree to which conditioning on the propensity score (via matching, weighting, or stratification) has balanced measured baseline covariates between exposure groups. Methods for balance assessment have been well described and are frequently implemented when using the propensity score with binary exposures. However, there is a paucity of information on how to assess baseline covariate balance when using the generalized propensity score. We describe how methods based on the standardized difference can be adapted for use with quantitative exposures when using the generalized propensity score. We also describe a method based on assessing the correlation between the quantitative exposure and each covariate in the sample when weighted using generalized propensity score -based weights. We conducted a series of Monte Carlo simulations to evaluate the performance of these methods. We also compared two different methods of estimating the generalized propensity score: ordinary least squared regression and the covariate balancing propensity score method. We illustrate the application of these methods using data on patients hospitalized with a heart attack with the quantitative exposure being creatinine level.

The use of propensity score methods with survival or time‐to‐event outcomes: reporting measures of effect similar to those used in randomized experiments

Propensity score methods are increasingly being used to estimate causal treatment effects in observational studies. In medical and epidemiological studies, outcomes are frequently time-to-event in nature. Propensity-score methods are often applied incorrectly when estimating the effect of treatment on time-to-event outcomes. This article describes how two different propensity score methods (matching and inverse probability of treatment weighting) can be used to estimate the measures of effect that are frequently reported in randomized controlled trials: (i) marginal survival curves, which describe survival in the population if all subjects were treated or if all subjects were untreated; and (ii) marginal hazard ratios. The use of these propensity score methods allows one to replicate the measures of effect that are commonly reported in randomized controlled trials with time-to-event outcomes: both absolute and relative reductions in the probability of an event occurring can be determined. We also provide guidance on variable selection for the propensity score model, highlight methods for assessing the balance of baseline covariates between treated and untreated subjects, and describe the implementation of a sensitivity analysis to assess the effect of unmeasured confounding variables on the estimated treatment effect when outcomes are time-to-event in nature. The methods in the paper are illustrated by estimating the effect of discharge statin prescribing on the risk of death in a sample of patients hospitalized with acute myocardial infarction. In this tutorial article, we describe and illustrate all the steps necessary to conduct a comprehensive analysis of the effect of treatment on time-to-event outcomes. © 2013 The authors. Statistics in Medicine published by John Wiley & Sons, Ltd.

Balancing continuous and categorical baseline covariates in sequential clinical trials using the area between empirical cumulative distribution functions

Covariate adaptive allocation is often adopted in sequential clinical trials to maintain the balance of baseline covariates that could potentially confound the outcome of a trial. Several allocation methods exist in the literature that can handle both continuous and categorical covariates. We propose a minimization approach to maintaining the balance of multiple continuous and categorical covariates in sequential clinical trials, which uses the area between the empirical cumulative distribution functions of the observed covariate values as the imbalance metric. Numerical results based on extensive simulation studies and a real dataset show that the proposed approach produces more accurate estimates of the treatment effect and leads to more powerful trials than the existing approaches for trials with binary, continuous, and time-to-event outcomes.

%GI: ASASMacro for Measuring and Testing Global Imbalance of Covariates within Subgroups

The global imbalance (GI) measure is a way for checking balance of baseline covariates that confound efforts to draw valid conclusions about treatment effects on outcomes of interest. In addition, GI is tested by means of a multivariate test. The GI measure and its test overcome some limitations of the common way for assessing the presence of imbalance in observed covariates that were discussed in D'Attoma and Camillo (2011). A user written SAS macro called %GI, to simultaneously measure and test global imbalance of baseline covariates is described. Furthermore, %GI also assesses global imbalance by subgroups obtained through several matching or classification methods (e.g., cluster analysis, propensity score subclassification, Rosenbaum and Rubin 1984), no matter how many groups are examined. %GI works with mixed categorical, ordinal and continuous covariates. Continuous baseline covariates need to be split into categories. It also works in the multi-treatment case. The use of the %GI macro will be illustrated using two artificial examples.

Quantifying the cost in power of ignoring continuous covariate imbalances in clinical trial randomization

Motivated by potentially serious imbalances of continuous baseline covariates in clinical trials, we investigated the cost in statistical power of ignoring the balance of these covariates in treatment allocation design for a logistic regression model. Based on data from a clinical trial of acute ischemic stroke treatment, computer simulations were used to create scenarios varying from the best possible baseline covariate balance to the worst possible imbalance, with multiple balance levels between the two extremes. The likelihood of each scenario occurring under simple randomization was evaluated. The power of the main effect test for treatment was examined. Our simulation results show that the worst possible imbalance is highly unlikely, but it can still occur under simple random allocation. Also, power loss could be nontrivial if balancing distributions of important continuous covariates were ignored even if adjustment is made in the analysis for important covariates. This situation, although unlikely, is more serious for trials with a small sample size and for covariates with large influence on primary outcome. These results suggest that attempts should be made to balance known prognostic continuous covariates at the design phase of a clinical trial even when adjustment is planned for these covariates at the analysis.

Goodness‐of‐fit diagnostics for the propensity score model when estimating treatment effects using covariate adjustment with the propensity score

The propensity score is defined to be a subject's probability of treatment selection, conditional on observed baseline covariates. Conditional on the propensity score, treated and untreated subjects have similar distributions of observed baseline covariates. In the medical literature, there are three commonly employed propensity-score methods: stratification (subclassification) on the propensity score, matching on the propensity score, and covariate adjustment using the propensity score. Methods have been developed to assess the adequacy of the propensity score model in the context of stratification on the propensity score and propensity-score matching. However, no comparable methods have been developed for covariate adjustment using the propensity score. Inferences about treatment effect made using propensity-score methods are only valid if, conditional on the propensity score, treated and untreated subjects have similar distributions of baseline covariates. We develop both quantitative and qualitative methods to assess the balance in baseline covariates between treated and untreated subjects. The quantitative method employs the weighted conditional standardized difference. This is the conditional difference in the mean of a covariate between treated and untreated subjects, in units of the pooled standard deviation, integrated over the distribution of the propensity score. The qualitative method employs quantile regression models to determine whether, conditional on the propensity score, treated and untreated subjects have similar distributions of continuous covariates. We illustrate our methods using a large dataset of patients discharged from hospital with a diagnosis of a heart attack (acute myocardial infarction). The exposure was receipt of a prescription for a beta-blocker at hospital discharge.

Assessing balance in measured baseline covariates when using many‐to‐one matching on the propensity‐score

The propensity score is defined to be a subject's probability of treatment selection, conditional on observed baseline covariates. Conditional on the propensity score, treated and untreated subjects have similar distributions of observed baseline covariates. Propensity-score matching is a commonly used propensity score method for estimating the effects of treatment on outcomes. Balance diagnostics have been previously described for use when 1:1 matching on the propensity score is employed. We illustrate that these methods can be misleading when many-to-one matching on the propensity score is employed. We then propose modifications of these methods that involve weighting each untreated subject by the inverse of the number of untreated subjects in the matched set. We describe both quantitative and qualitative methods to assess the balance in baseline covariates between treated and untreated subjects in a sample obtained by many-to-one matching on the propensity score. The quantitative method uses the weighted standardized difference. The qualitative methods employ graphical methods to compare the distribution of continuous baseline covariates between treated and untreated subjects in the weighted sample. We illustrate our methods using a large sample of patients discharged from hospital with a diagnosis of a heart attack (acute myocardial infarction). The exposure was receipt of a prescription for a statin at hospital discharge.

Primer on Statistical Interpretation or Methods Report Card on Propensity-Score Matching in the Cardiology Literature From 2004 to 2006

Propensity-score matching is frequently used in the cardiology literature. Recent systematic reviews have found that this method is, in general, poorly implemented in the medical literature. The study objective was to examine the quality of the implementation of propensity-score matching in the general cardiology literature. A total of 44 articles published in the American Heart Journal, the American Journal of Cardiology, Circulation, the European Heart Journal, Heart, the International Journal of Cardiology, and the Journal of the American College of Cardiology between January 1, 2004, and December 31, 2006, were examined. Twenty of the 44 studies did not provide adequate information on how the propensity-score-matched pairs were formed. Fourteen studies did not report whether matching on the propensity score balanced baseline characteristics between treated and untreated subjects in the matched sample. Only 4 studies explicitly used statistical methods appropriate for matched studies to compare baseline characteristics between treated and untreated subjects. Only 11 (25%) of the 44 studies explicitly used statistical methods appropriate for the analysis of matched data when estimating the effect of treatment on the outcomes. Only 2 studies described the matching method used, assessed balance in baseline covariates by appropriate methods, and used appropriate statistical methods to estimate the treatment effect and its significance. Application of propensity-score matching was poor in the cardiology literature. Suggestions for improving the reporting and analysis of studies that use propensity-score matching are provided.