Unbiased Estimates Of Treatment Effects Research Articles

Design issues in crossover trials involving patients with Parkinson's disease.

Crossover designs are frequently used to assess treatments for patients with Parkinson's disease. Typically, two-period two-treatment trials include a washout period between the 2 periods and assume that the washout period is sufficiently long to eliminate carryover effects. A complementary strategy might be to jointly model carryover and treatment effects, though this has rarely been done in Parkinson's disease crossover studies. The primary objective of this research is to demonstrate a modeling approach that assesses treatment and carryover effects in one unified mixed model analysis and to examine how it performs in a simulation study and a real data analysis example, as compared to other data analytic approaches used in Parkinson's disease crossover studies. We examined how three different methods of analysis (standard crossover t-test, mixed model with a carryover term included in model statement, and mixed model with no carryover term) performed in a simulation study and illustrated the methods in a real data example in Parkinson's disease. The simulation study based on the presence of a carryover effect indicated that mixed models with a carryover term and an unstructured correlation matrix provided unbiased estimates of treatment effect and appropriate type I error. The methods are illustrated in a real data example involving Parkinson's disease. Our literature review revealed that a majority of crossover studies included a washout period but did not assess whether the washout was sufficiently long to eliminate the possibility of carryover. We recommend using a mixed model with a carryover term and an unstructured correlation matrix to obtain unbiased estimates of treatment effect.

Covariate Balance for Observational Effectiveness Studies: A Comparison of Matching and Weighting

Propensity score matching and weighting methods are often used in observational effectiveness studies to reduce imbalance between treated and untreated groups on a set of potential confounders. However, much of the prior methodological literature on matching and weighting has yet to examine performance for scenarios with a majority of treated units, as is often encountered with programs and interventions that have been widely disseminated or “scaled-up.” Using a series of Monte Carlo simulations, we compare the performance of k:1 matching with replacement and weighting methods with respect to covariate balance, bias, and mean squared error. Results indicate that the accuracy of all methods declined as treatment prevalence increased. While weighting produced the largest reduction in covariate imbalance, 1:1 matching with replacement provided the most unbiased treatment effect estimates. An applied example using empirical school-level data is provided to further illustrate the application and interpretation of these methods to a real-world scale-up effort. We conclude by considering the implications of propensity score methods for observational effectiveness studies with a particular focus on educational research.

P-157 Embryo culture under high humidity conditions in time lapse system does not improve ongoing pregnancy rates: a retrospective propensity score model study 299 ICSI cycles

Abstract Study question Does culture in high relative humidity conditions (HC) improve ongoing pregnancy rates when using a time-lapse system (TLS) for embryo culture? Summary answer Using The use of a TLS under HC does not increase the likelihood a pregnancy in comparison with cultured in dry condition (DC). What is known already Factors potentially affecting embryo development demand precise control in the IVF laboratory environment, especially for correct incubator usage. TLS provide stability during embryo culture, which is a well-known key factor for undisturbed embryo development. The humidity content of embryo culture atmosphere is especially relevant to prevention of oscillations in culture media osmolality. Previous data suggest that culture under HC has a significant effect on embryo quality and morphokinetics. However, studies assessing the effect of HC on clinical outcome are rare and inconclusive. Study design, size, duration A retrospective study of 299 women undergoing their first ICSI cycle from April 2021 to December 2021, of whom patients assigned to dry or humid conditions were 184 and 115, respectively. Participants/materials, setting, methods We used a GERI incubator, with 6 separated chambers for individual patients (3 HC, 3 DC). The effect of HC on ongoing pregnancy rate was assessed by the propensity matched sample, to reduce potential differences between women undergoing either HC or DC and to obtain unbiased estimation of treatment effect. Variables included in the propensity score model were female age, AMH, FSH, BMI, partner’s age, sperm quality and number of embryos transferred. Main results and the role of chance Patient demographics were homogeneously distributed in the two study groups. After adjusting for several confounding variables and applying the Propensity Score, no statistically differences were observed for embryo quality on day 3 [51,64 ± 34,33 (DC) Vs 53,83 ± 30,71 (HC)], blastulation rate [45,5% ± 22,60 (DC) Vs 45,9 ± 22,80 (HC)] and ongoing pregnancy rate. A trend was observed of increased ORs for ongoing pregnancy in HC conditions [OR = 1.56, 95% CI (0,77-3,15), p=0.21]. Limitations, reasons for caution The study is limited by its retrospective design, although a propensity score model may mitigate such a limitation. The lack of data on live birth rate is an additional limitation. Wider implications of the findings These results suggest that HC conditions does not increases the likelihood of embryos to achieve an ongoing pregnancy with respect to those cultured in dry conditions. Further data collection and analysis are ongoing to investigate more thoroughly and extended clinical outcome results. Trial registration number N.A.

Multiple imputation for longitudinal data using Bayesian lasso imputation model.

Multiple imputation is a promising approach to handle missing data and is widely used in analysis of longitudinal clinical studies. A key consideration in the implementation of multiple imputation is to obtain accurate imputed values by specifying an imputation model that incorporates auxiliary variables potentially associated with missing variables. The use of informative auxiliary variables is known to be beneficial to make the missing at random assumption more plausible and help to reduce uncertainty of the imputations; however, it is not straightforward to pre-specify them in many cases. We propose a data-driven specification of the imputation model using Bayesian lasso in the context of longitudinal clinical study, and develop a built-in function of the Bayesian lasso imputation model which is performed within the framework of multiple imputation using chained equations. A simulation study suggested that the Bayesian lasso imputation model worked well in a variety of longitudinal study settings, providing unbiased treatment effect estimates with well-controlled type I error rates and coverage probabilities of the confidence interval; in contrast, ignorance of the informative auxiliary variables led to serious bias and inflation of type I error rate. Moreover, the Bayesian lasso imputation model offered higher statistical powers compared with conventional imputation methods. In our simulation study, the gains in statistical power were remarkable when the sample size was small relative to the number of auxiliary variables. An illustration through a real example also suggested that the Bayesian lasso imputation model could give smaller standard errors of the treatment effect estimate.

Methods for population adjustment with limited access to individual patient data: A review and simulation study.

Population-adjusted indirect comparisons estimate treatment effects when access to individual patient data is limited and there are cross-trial differences in effect modifiers. Popular methods include matching-adjusted indirect comparison (MAIC) and simulated treatment comparison (STC). There is limited formal evaluation of these methods and whether they can be used to accurately compare treatments. Thus, we undertake a comprehensive simulation study to compare standard unadjusted indirect comparisons, MAIC and STC across 162 scenarios. This simulation study assumes that the trials are investigating survival outcomes and measure continuous covariates, with the log hazard ratio as the measure of effect. MAIC yields unbiased treatment effect estimates under no failures of assumptions. The typical usage of STC produces bias because it targets a conditional treatment effect where the target estimand should be a marginal treatment effect. The incompatibility of estimates in the indirect comparison leads to bias as the measure of effect is non-collapsible. Standard indirect comparisons are systematically biased, particularly under stronger covariate imbalance and interaction effects. Standard errors and coverage rates are often valid in MAIC but the robust sandwich variance estimator underestimates variability where effective sample sizes are small. Interval estimates for the standard indirect comparison are too narrow and STC suffers from bias-induced undercoverage. MAIC provides the most accurate estimates and, with lower degrees of covariate overlap, its bias reduction outweighs the loss in precision under no failures of assumptions. An important future objective is the development of an alternative formulation to STC that targets a marginal treatment effect.

Unemployment and Job Change Affect Future Time Perspective

This paper examines the effects of workers becoming unemployed, unemployment duration, and job change as antecedents on their future time perspective (FTP). Drawing on socioemotional selectivity theory, unemployment theories, and job mobility theory, we hypothesize that becoming unemployed, unemployment duration, and job change to be associated with FTP. We apply propensity score matching to control for potentially confounding covariates and to gain unbiased estimates of treatment effects in a quasi-experimental study comprising 1.261 participants of the German Socioeconomic Panel (SOEP) providing longitudinal data from ten survey waves between 2007 and 2017. The results provide support for effects of unemployment experience and job change on future time perspective.

Confounding and regression adjustment in difference-in-differences studies.

ObjectiveTo define confounding bias in difference‐in‐difference studies and compare regression‐ and matching‐based estimators designed to correct bias due to observed confounders.Data sourcesWe simulated data from linear models that incorporated different confounding relationships: time‐invariant covariates with a time‐varying effect on the outcome, time‐varying covariates with a constant effect on the outcome, and time‐varying covariates with a time‐varying effect on the outcome. We considered a simple setting that is common in the applied literature: treatment is introduced at a single time point and there is no unobserved treatment effect heterogeneity.Study designWe compared the bias and root mean squared error of treatment effect estimates from six model specifications, including simple linear regression models and matching techniques.Data collectionSimulation code is provided for replication.Principal findingsConfounders in difference‐in‐differences are covariates that change differently over time in the treated and comparison group or have a time‐varying effect on the outcome. When such a confounding variable is measured, appropriately adjusting for this confounder (ie, including the confounder in a regression model that is consistent with the causal model) can provide unbiased estimates with optimal SE. However, when a time‐varying confounder is affected by treatment, recovering an unbiased causal effect using difference‐in‐differences is difficult.ConclusionsConfounding in difference‐in‐differences is more complicated than in cross‐sectional settings, from which techniques and intuition to address observed confounding cannot be imported wholesale. Instead, analysts should begin by postulating a causal model that relates covariates, both time‐varying and those with time‐varying effects on the outcome, to treatment. This causal model will then guide the specification of an appropriate analytical model (eg, using regression or matching) that can produce unbiased treatment effect estimates. We emphasize the importance of thoughtful incorporation of covariates to address confounding bias in difference‐in‐difference studies.

Logistic or linear? Estimating causal effects of experimental treatments on binary outcomes using regression analysis.

When the outcome is binary, psychologists often use nonlinear modeling strategies such as logit or probit. These strategies are often neither optimal nor justified when the objective is to estimate causal effects of experimental treatments. Researchers need to take extra steps to convert logit and probit coefficients into interpretable quantities, and when they do, these quantities often remain difficult to understand. Odds ratios, for instance, are described as obscure in many textbooks (e.g., Gelman & Hill, 2006, p. 83). I draw on econometric theory and established statistical findings to demonstrate that linear regression is generally the best strategy to estimate causal effects of treatments on binary outcomes. Linear regression coefficients are directly interpretable in terms of probabilities and, when interaction terms or fixed effects are included, linear regression is safer. I review the Neyman-Rubin causal model, which I use to prove analytically that linear regression yields unbiased estimates of treatment effects on binary outcomes. Then, I run simulations and analyze existing data on 24,191 students from 56 middle schools (Paluck, Shepherd, & Aronow, 2013) to illustrate the effectiveness of linear regression. Based on these grounds, I recommend that psychologists use linear regression to estimate treatment effects on binary outcomes. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Effect of Caesarean Section on Neonatal Health in Bangladesh after Controlling Selection Bias: Propensity Score Based Analysis

The paper examines the effect of caesarean section (C-section) on early neonatal mortality, neonatal mortality, and early initiation of breastfeeding using Bangladesh Demographic and Health Survey (BDHS), 2014 data. Propensity score matching and weighting methods were used to estimate unbiased estimate of treatment effect. The study demonstrates how conclusion about treatment effect varies with and without having balance in the treatment groups. Standard analysis, without caring about balance, reveals that C-section has no significant impact on early neonatal mortality and neonatal mortality. After applying propensity score adjusted methods, balance was achieved in the treatment groups and it was found that C-section has significant effect on early neonatal mortality and neonatal mortality. However, there was no difference between standard and PS adjusted methods in estimating the effect of C-section on early initiation of breastfeeding. It is concluded that children who were delivered by C-section have significantly lower odds of early neonatal mortality, neonatal mortality, and early initiation of breastfeeding as compared to the children who were not delivered by C-section.
 Dhaka Univ. J. Sci. 69(1): 23-29, 2021 (January)

Impact of Medicaid expansion on women with gynecologic cancer: a difference-in-difference analysis

Women with gynecologic cancer face socioeconomic disparities in care that affect survival outcomes. The Affordable Care Act offered states the option to expand Medicaid enrollment eligibility criteria as a means of improving timely and affordable access to care for the most vulnerable. The variable uptake of expansion by states created a natural experiment, allowing for quasi-experimental methods that offer more unbiased estimates of treatment effects from retrospective data than the traditional regression adjustment. To use a quasi-experimental, difference-in-difference framework to create unbiased estimates of impact of Medicaid expansion on women with gynecologic cancer. We performed a quasi-experimental retrospective cohort study from the National Cancer Database files for women with invasive cancers of the uterus, ovary and fallopian tube, cervix, vagina, and vulva diagnosed from 2008 to 2016. Using a marker for state Medicaid expansion status, we created difference-in-difference models to assess the impact of Medicaid expansion on the outcomes of access to and timeliness of care. We excluded women aged <40 years owing to the suppression of the state Medicaid expansions status in the data and women aged ≥65 years owing to the universal Medicare coverage availability. Our primary outcome was the rate of uninsurance at diagnosis. Secondary outcomes included Medicaid coverage, early-stage diagnosis, treatment at an academic facility, and any treatment or surgery within 30 days of diagnosis. Models were run within multiple subgroups and on a propensity-matched cohort to assess the robustness of the treatment estimates. The assumption of parallel trends was assessed with event study time plots. Our sample included 335,063 women. Among this cohort, 121,449 were from nonexpansion states and 213,614 were from expansion states, with 79,886 posttreatment cases diagnosed after the expansion took full effect in expansion states. The groups had minor differences in demographics, and we found occasional preperiod event study coefficients diverging from the mean, but the outcome trends were generally similar between the expansion and nonexpansion states in the preperiod, satisfying the necessary assumption for the difference-in-difference analysis. In a basic difference-in-difference model, the Medicaid expansion in January 2014 was associated with significant increases in insurance at diagnosis, treatment at an academic facility, and treatment within 30 days of diagnosis (P<.001 for all). In an adjusted model including all states and accounting for variable expansion implementation time, there was a significant treatment effect of Medicaid expansion on the reduction in uninsurance at diagnosis (-2.00%; 95% confidence interval,-2.3 to-1.7; P<.001), increases in early-stage diagnosis (0.80%; 95% confidence interval, 0.2-1.4; P=.02), treatment at an academic facility (0.83%; 95% confidence interval, 0.1-1.5; P=.02), treatment within 30 days (1.62%; 95% confidence interval, 1.0-2.3; P<.001), and surgery within 30 days (1.54%; 95% confidence interval, 0.8-2.3; P<.001). In particular, large gains were estimated for women living in low-income zip codes, Hispanic women, and women with cervical cancer. Estimates from the subgroup and propensity-matched cohorts were generally consistent for all outcomes besides early-stage diagnosis and treatment within 30 days. Medicaid expansion was significantly associated with gains in the access and timeliness of treatment for nonelderly women with gynecologic cancer. The implementation of Medicaid expansion could greatly benefit women in nonexpansion states. Gynecologists and gynecologic oncologists should advocate for Medicaid expansion as a means of improving outcomes and reducing socioeconomic and racial disparities.

The inverse-probability-of-censoring weighting (IPCW) adjusted win ratio statistic: an unbiased estimator in the presence of independent censoring

ABSTRACT The win ratio method has received much attention in methodological research, ad hoc analyses, and designs of prospective studies. As the primary analysis, it supported the approval of tafamidis for the treatment of cardiomyopathy to reduce cardiovascular mortality and cardiovascular-related hospitalization. However, its dependence on censoring is a potential shortcoming. In this article, we propose the inverse-probability-of-censoring weighting (IPCW) adjusted win ratio statistic (i.e., the IPCW-adjusted win ratio statistic) to overcome censoring issues. We consider independent censoring, common censoring across endpoints, and right censoring. We develop an asymptotic variance estimator for the logarithm of the IPCW-adjusted win ratio statistic and evaluate it via simulation. Our simulation studies show that, as the amount of censoring increases, the unadjusted win proportions may decrease greatly. Consequently, the bias of the unadjusted win ratio estimate may increase greatly, producing either an overestimate or an underestimate. We demonstrate theoretically and through simulation that the IPCW-adjusted win ratio statistic gives an unbiased estimate of treatment effect.

Accounting for Latent Covariates in Average Effects from Count Regressions

The effectiveness of a treatment on a count outcome can be assessed using a negative binomial regression, where treatment effects are defined as the difference between the expected outcome under treatment and under control. These treatment effects can to date only be estimated if all covariates are manifest (observed) variables. However, some covariates are latent variables that are measured by multiple fallible indicators. In such cases, it is important to control for measurement error of covariates in order to avoid attenuation bias and to get unbiased treatment effect estimates. In this paper, we propose a new approach to compute average and conditional treatment effects in regression models with a logarithmic link function involving multiple latent and manifest covariates. We extend the previously presented moment-based approach in several aspects: Building on a multigroup SEM framework for count variables instead of the generalized linear model, we allow for latent covariates and multiple covariates. We provide an illustrative example to explain the application and estimation in structural equation modeling software.

Assessing causal treatment effect estimation when using large observational datasets

BackgroundRecently, there has been a heightened interest in developing and evaluating different methods for analysing observational data. This has been driven by the increased availability of large data resources such as Electronic Health Record (EHR) data alongside known limitations and changing characteristics of randomised controlled trials (RCTs). A wide range of methods are available for analysing observational data. However, various, sometimes strict, and often unverifiable assumptions must be made in order for the resulting effect estimates to have a causal interpretation. In this paper we will compare some common approaches to estimating treatment effects from observational data in order to highlight the importance of considering, and justifying, the relevant assumptions prior to conducting an observational analysis.MethodsA simulation study was conducted based upon a small cohort of patients with chronic obstructive pulmonary disease. Two-stage least squares instrumental variables, propensity score, and linear regression models were compared under a range of different scenarios including different strengths of instrumental variable and unmeasured confounding. The effects of violating the assumptions of the instrumental variables analysis were also assessed. Sample sizes of up to 200,000 patients were considered.ResultsTwo-stage least squares instrumental variable methods can yield unbiased treatment effect estimates in the presence of unmeasured confounding provided the sample size is sufficiently large. Adjusting for measured covariates in the analysis reduces the variability in the two-stage least squares estimates. In the simulation study, propensity score methods produced very similar results to linear regression for all scenarios. A weak instrument or strong unmeasured confounding led to an increase in uncertainty in the two-stage least squares instrumental variable effect estimates. A violation of the instrumental variable assumptions led to bias in the two-stage least squares effect estimates. Indeed, these were sometimes even more biased than those from a naïve linear regression model.ConclusionsInstrumental variable methods can perform better than naïve regression and propensity scores. However, the assumptions need to be carefully considered and justified prior to conducting an analysis or performance may be worse than if the problem of unmeasured confounding had been ignored altogether.

A note on misspecification in general linear models with correlated errors for the analysis of crossover clinical trials.

Among various approaches to the repeated measures analysis in crossover clinical trials, the general linear models (GLMs) with correlated errors attract substantial attention due to their simplicity in model specification, implementation, and interpretation. The goal of this research note is to conduct simulation studies to numerically investigate the impact of model misspecification in the GLMs with correlated errors in the analysis of crossover trials. A series of synthetic two-treatment and three-treatment crossover trials were designed, and simulation studies were conducted to assess how treatment effect estimation, type I error rates, and power can be affected by misspecified period effects, carryover effects, and variance-covariance structures in the GLMs. Numerical studies confirm that (i) the GLMs with terms for both carryover and period effects and with an unstructured variance-covariance matrix can provide unbiased treatment effect estimates and control of Type I error rates and that (ii) misspecification in either period effects, carryover effects, or covariance structures in the GLMs can induce inflated type I error, declined power, or biased treatment effect estimates. Although methodologic contribution of this research note is minimal, we provide practical recommendations and advice to pharmaceutical sponsors and other investigational drugs and device applicants in designing and analyzing crossover trials using the GLMs with correlated errors.

The Effect Of The Hospital Readmissions Reduction Program On Readmission And Observation Stay Rates For Heart Failure.

The Hospital Readmissions Reduction Program reduces Medicare prospective payments for hospitals with excess readmissions for selected diagnoses. By comparing data for patients who were readmitted or placed on observation status immediately before and immediately after the thirty-day cutoff for penalties, we sought to determine whether hospitals have responded to the program by shifting readmissions for heart failure to observation status. We used regression discontinuity, taking advantage of the cutoff to generate unbiased estimates of treatment effects. Overall, we found no evidence that the program has affected the use of observation stays. However, for nonpenalized hospitals, the use of observation status was 5.4percent higher for patients returning to the hospital immediately before the thirty-day cutoff than for patients returning immediately after the cutoff, which suggests that some hospitals may have used observation status to help avoid penalties. Because differences in the cost-sharing rules may lead to higher out-of-pocket expenses for Medicare patients placed on observation status, the program could have an inequitable financial impact.

PRM116 - Do Models With Better Goodness Of Fit Ensure Less Biased Treatment Effect Estimates?

There is widespread misconception that goodness-of-fit statistics are critical in determining whether treatment effects estimated from a model are unbiased. We aimed to examine whether models with better goodness-of-fit ensure less biased treatment effect estimates. We constructed a simulated dataset of 10,000 patients to compare two treatments on a continuous outcome of a prognostic score (mean: 0.82; range:-6.2to8.0) wherein a higher score indicates a higher risk of mortality within one year. By design in our simulation, the new treatment would lower the prognostic score by 0.1 compared to the traditional treatment. We then generated two study designs using this dataset: 1) a randomized controlled trial (RCT) with randomly assigned treatment; 2) an observational study design wherein patients receiving the new treatment tended to be sicker on observed (e.g. older age, higher comorbidity score) and unobserved covariates. Treatment effect estimates were obtained from the observational study design using: 1) multivariable linear regression accounting for observed covariates, and 2) instrumental variable two-stage least squares (2SLS) regression accounting for observed and unobserved confounders and were compared with 3) the regression results from the RCT. Bootstrapping with 1000 resamples and standardized errors (SE) were used to estimate out-of-sample prediction for each of the three models. Across the three models, multivariable linear regression had the highest goodness-of-fit (adjusted R-squared 0.215) and best out-of-sample prediction (SE: 2.15), but very biased treatment effect estimates due to unmeasured confounders (bias: 1.119, 95%CI: 1.101to1.137). The RCT regression had the lowest R-squared (0.001) and 2SLS had a R-squared of 0.122. Both RCT (bias: 0.003, 95%CI: -0.017to0.024) and 2SLS (bias: 0.004, 95%CI: -0.039to0.046) had unbiased estimates. Models with better goodness-of-fit would have better predictive power but do not ensure less biased treatment effect estimates. Correct study design and model specification are important for unbiased treatment effect estimates.

Statistical primer: propensity score matching and its alternatives.

Propensity score (PS) methods offer certain advantages over more traditional regression methods to control for confounding by indication in observational studies. Although multivariable regression models adjust for confounders by modelling the relationship between covariates and outcome, the PS methods estimate the treatment effect by modelling the relationship between confounders and treatment assignment. Therefore, methods based on the PS are not limited by the number of events, and their use may be warranted when the number of confounders is large, or the number of outcomes is small. The PS is the probability for a subject to receive a treatment conditional on a set of baseline characteristics (confounders). The PS is commonly estimated using logistic regression, and it is used to match patients with similar distribution of confounders so that difference in outcomes gives unbiased estimate of treatment effect. This review summarizes basic concepts of the PS matching and provides guidance in implementing matching and other methods based on the PS, such as stratification, weighting and covariate adjustment.

Using Bayesian Correspondence Criteria to Compare Results From a Randomized Experiment and a Quasi-Experiment Allowing Self-Selection.

Randomized experiments yield unbiased estimates of treatment effect, but such experiments are not always feasible. So researchers have searched for conditions under which randomized and nonrandomized experiments can yield the same answer. This search requires well-justified and informative correspondence criteria, that is, criteria by which we can judge if the results from an appropriately adjusted nonrandomized experiment well-approximate results from randomized experiments. Past criteria have relied exclusively on frequentist statistics, using criteria such as whether results agree in sign or statistical significance or whether results differ significantly from each other. In this article, we show how Bayesian correspondence criteria offer more varied, nuanced, and informative answers than those from frequentist approaches. We describe the conceptual bases of Bayesian correspondence criteria and then illustrate many possibilities using an example that compares results from a randomized experiment to results from a parallel nonequivalent comparison group experiment in which participants could choose their condition. Results suggest that, in this case, the quasi-experiment reasonably approximated the randomized experiment. We conclude with a discussion of the advantages (computation of relevant quantities, interpretation, and estimation of quantities of interest for policy), disadvantages, and limitations of Bayesian correspondence criteria. We believe that in most circumstances, the advantages of Bayesian approaches far outweigh the disadvantages.

On Measuring and Reducing Selection Bias With a Quasi‐Doubly Randomized Preference Trial

AbstractRandomized experiments provide unbiased estimates of treatment effects, but are costly and time consuming. We demonstrate how a randomized experiment can be leveraged to measure selection bias by conducting a subsequent observational study that is identical in every way except that subjects choose their treatment—a quasi‐doubly randomized preference trial (quasi‐DRPT). Researchers first strive to think of and measure all possible confounders and then determine how well these confounders as controls can reduce or eliminate selection bias. We use a quasi‐DRPT to study the effect of class time on student performance in an undergraduate introductory microeconomics course at a large public university, illustrating its required design elements: experimental and choice arms conducted in the same setting with identical interventions and measurements, and all confounders measured prospectively to treatment assignment or choice. Quasi‐DRPTs augment randomized experiments in real‐world settings where participants choose their treatments.

Methods for Analysis of Pre-Post Data in Clinical Research: A Comparison of Five Common Methods.

Often repeated measures data are summarized into pre-post-treatment measurements. Various methods exist in the literature for estimating and testing treatment effect, including ANOVA, analysis of covariance (ANCOVA), and linear mixed modeling (LMM). Under the first two methods, outcomes can either be modeled as the post treatment measurement (ANOVA-POST or ANCOVA-POST), or a change score between pre and post measurements (ANOVA-CHANGE, ANCOVA-CHANGE). In LMM, the outcome is modeled as a vector of responses with or without Kenward-Rogers adjustment. We consider five methods common in the literature, and discuss them in terms of supporting simulations and theoretical derivations of variance. Consistent with existing literature, our results demonstrate that each method leads to unbiased treatment effect estimates, and based on precision of estimates, 95% coverage probability, and power, ANCOVA modeling of either change scores or post-treatment score as the outcome, prove to be the most effective. We further demonstrate each method in terms of a real data example to exemplify comparisons in real clinical context.