Bias In Treatment Effect Research Articles

Has the opening of high-speed railway in China promoted urban low-carbon transformation along the route: re-estimation based on staggered DID robust estimators.

Considering the opening of high-speed railway (HSR) as a quasi-natural experiment, this paper explores the emission reduction effect of HSR opening and its mechanisms and heterogeneity characteristics based on the panel data of 272 cities in China from 2006 to 2019 by staggered DID model and the mediating effect model. It is found that (1) the opening of HSR significantly reduces the carbon emission level of cities, and the conclusion still holds after controlling the endogeneity issue and a series of robustness tests; (2) Bancon decomposition and negative weight diagnostics both show that the estimated average treatment effect bias is small and the robust estimators are more robust; (3) by reducing production costs and promoting inter-regional industrial division of labor, the opening of HSR promotes intra-industrial structural transformation and inter-industrial structural transformation, respectively, thus reducing urban carbon emissions. At the same time, HSR achieves urban carbon emission reduction by promoting free movement of population and human capital agglomeration; and (4) further study finds that there is regional heterogeneity in the emission reduction effect of HSR opening, and the carbon reduction effect is mainly reflected in large and medium-sized cities, near provincial capitals, non-resource-based cities, and cities with rich educational resources, which indicates that the location of future HSR should be combined with the city's own characteristics and be tailored to the city.

Voluntary environmental regulations, greenwashing and green innovation: Empirical study of China's ISO14001 certification

Voluntary environmental regulations play a crucial role in promoting green innovation. However, the existing literature on the subject has yielded inconsistent conclusions. To address this issue, this study utilizes data from China's list companies between 2008 and 2020, employing both the difference-in-differences (DID) and instrumental variable methods to accurately estimate the impact of ISO14001 certification on green innovation. Our findings indicate that ISO14001 certification enhances both the quantity and quality of green innovations. Furthermore, our results remain robust even after implementing the propensity score matching (PSM) method and conducting placebo tests. This paper also excludes heterogeneous treatment effect bias in staggered DID. In addition, the impact of ISO14001 on green innovation is focused on green innovation related to source control. The study also considers the impact of greenwashing and ownership on the relationship between ISO14001 certification and green innovation. Greenwashing is found to have a negative impact on green innovation, especially in terms of quality, and enterprises with more greenwashing tend to have a weaker response to ISO14001 certification. Finally, our mechanism test indicates that ISO14001-induced green technology innovation serves as a leverage effect on existing innovation activities, rather than a crowding-out effect on other technological innovations or a speculative activity aimed at defrauding government subsidies.

Examination of early treatment effects and related biases during the conduct of two UK-wide pragmatic orthopaedic surgical trials: ProFHER and UK FROST.

Early large treatment effects can arise in small studies, which lessen as more data accumulate. This study aimed to retrospectively examine whether early treatment effects occurred for two multicentre orthopaedic randomized controlled trials (RCTs) and explore biases related to this. Included RCTs were ProFHER (PROximal Fracture of the Humerus: Evaluation by Randomisation), a two-arm study of surgery versus non-surgical treatment for proximal humerus fractures, and UK FROST (United Kingdom Frozen Shoulder Trial), a three-arm study of two surgical and one non-surgical treatment for frozen shoulder. To determine whether early treatment effects were present, the primary outcome of Oxford Shoulder Score (OSS) was compared on forest plots for: the chief investigator's (CI) site to the remaining sites, the first five sites opened to the other sites, and patients grouped in quintiles by randomization date. Potential for bias was assessed by comparing mean age and proportion of patients with indicators of poor outcome between included and excluded/non-consenting participants. No bias in treatment effect was observed overall for the CI site, or the first five sites, compared with the remaining sites in either trial. An early treatment effect on the OSS was observed for the first quintile of participants recruited to ProFHER only (clinically relevant difference of seven points). Selection bias for age was observed in the ProFHER trial only, with slightly younger patients being recruited into the study. Both trials showed some selection bias for markers of poor prognosis, although these did not appear to change over time. No bias in treatment effects overall were found at the CI or early sites set-up. An early treatment effect was found in one of the two trials, which was likely a chance effect as this did not continue during the study. Selection bias was observed in both RCTs, however this was minimal and did not impact on outcome.

On the causality and plausibility of treatment effects in operations management research

Empirical research in operations management (OM) has made rapid strides in the last 30 years, and increasingly, OM researchers are leveraging methods used in the econometrics and statistics literature to assess the causal effects of interventions. We discuss the two key challenges in assessing causality with observational data (i.e., baseline bias, differential treatment effect bias) and how dominant identification approaches such as matching, instrumental variables, regression discontinuity, difference‐in‐differences, and fixed effects deal with such challenges. We surface the key underlying assumptions of different causal estimation methods and discuss how OM scholars have used these methods in the last few years. We hope that reflecting on the plausibility and substantive meaning of underlying assumptions regarding different identification strategies in a particular context will lead to a better conceptualization, execution, evaluation, dissemination, and consumption of OM research. We conclude with a few thoughts that authors and reviewers may find helpful in their research as they engage in discourse related to causality.

Independent prognostic value of ventricular premature complexes during exercise and recovery in asymptomatic patients: a meta-analysis of observational studies

Abstract Introduction Ventricular premature contractions (VPCs) are a common finding during a cardiac stress test. The independent prognostic value of these findings in asymptomatic participants is not clear. Purpose To investigate the cumulative independent prognostic value of VPC for adverse outcomes. Methods We conducted a systematic review and meta-analysis of observational studies in the following databases: MEDLINE, Embase®, and Cochrane Central from inception till January 28, 2022. The primary outcome was the cumulative hazard of all-cause mortality with VPCs during exercise or recovery in asymptomatic participants. The secondary outcome was cardiovascular mortality. We stratified results based on VPC during exercise or recovery. We included studies with participants with no known CV disease and known CV disease. If there was known CV disease at baseline, the study should have adjusted for at least ≥1 known confounder. If a study reported low risk or higher risk VPCs, we included outcomes with higher risk VPCs to make the data comparable across literature. Random effect meta-analyses were used to predict cumulative hazard ratios. The review was registered with PROSPERO (registration number CRD42022297028) Results We found 10 studies with 47,497 participants that met our inclusion criteria (8 studies with all-cause and CV mortality, 2 studies with CV mortality only). Three studies included patients with baseline CV disease (proportion 12–33%). Follow-up duration ranged from 5.3 to 20.3 years. Definition of VPCs differed in the individual studies. Both VPCs during exercise and recovery were associated with a higher hazard of all-cause mortality (HR 1.21, 95% confidence interval [CI] 1.07, 1.34 and 1.32, 95% CI 1.06, 1.58, p<0.001 and low heterogeneity for both, respectively, Figure 1A). There was no small study treatment effect bias. Similarly, both VPCs during exercise and recovery were associated with a higher hazard of CV mortality (HR 1.63, 95% CI 1.30, 1.96 and 1.68, 95% CI 1.00, 2.35, p<0.001, and low heterogeneity for both, respectively, Figure 1B). The presence of VPC had a numerically stronger association with CV mortality than all-cause mortality. VPCs during recovery seem to have more prognostic value than VPC during rest. Conclusion After adjusting for known confounders, VPCs during exercise and recovery predict a higher risk of all-cause and CV mortality. The risk is higher with VPCs during recovery. Funding Acknowledgement Type of funding sources: None.

The impact of ignoring Interval censoring in progression-free survival in cancer trials: a systematic review

Introduction & Objective: From statistical literature, the bias in treatment effect from ignoring interval censoring in Progression-free survival (PFS) is demonstrated. However, the impact on estimators caused by interval censoring is not carefully took account and investigated by researchers in practice. The objective of this study is to examine the impact of accounting for interval censoring in practice among RCTs used to support FDA approvals anti-cancer drugs between the years 2005 and 2019 that used PFS as an endpoint.
 Methods: In this systematic review, the differences of hazard ratios between two methods: considering and ignoring interval censoring, are visualized by Kaplan-Meier survival curves and estimated from a Cox proportional hazard model of 87 RCTs. With assumption that these differences and mean differences (bias) follow a normal distribution, limits of agreement of differences and confidence interval of bias are used to represent agreement of two methods.
 Results: Limits of agreement of difference range from -0.044 to 0.0615, while confidence intervals for the bias range from 0.0026 to 0.0145, which does not include zero, resulting in estimated treatment effect differs for two methods.
 Conclusion: In general, bias caused by interval censoring in treatment effect exists with large sample studies. Focusing on individual clinical trials, limits of agreement can provide more information for researchers to make decision on how to account for interval censoring.

Addressing Bias in Responder Analysis of Patient-Reported Outcomes

IntroductionQuantitative patient-reported outcome (PRO) measures ideally are analyzed on their original scales and responder analyses are used to aid the interpretation of those primary analyses. As stated in the FDA PRO Guidance for Medical Product Development (2009), one way to lend meaning and interpretation to such a PRO measure is to dichotomize between values where within-patient changes are considered clinically important and those that are not. But even a PRO scale with a cutoff score that discriminates well between responder and non-responders is fraught with some misclassification.MethodsUsing estimates of sensitivity and specificity on classification of responder status from a PRO instrument, formulas are provided to correct for such responder misclassification under the assumption of no treatment misclassification. Two case studies from sexual medicine illustrate the methodology.ResultsAdjustment formulas on cell counts for responder misclassification are a direct extension of correction formulas for misclassification on disease from a two-way cross-classification table of disease (yes, no) and exposure (yes, no). Unadjusted and adjusted estimates of treatment effect are compared in terms of odds ratio, response ratio, and response difference. In the two case studies, there was considerable underestimation of treatment effect.Discussion and conclusionsThe methodology can be applied to different therapeutic areas. Limitations of the methodology, such as when adjusted cell estimates become negative, are highlighted. The role of anchor-based methodology is discussed for obtaining estimates of sensitivity and specificity on responder classification. Correction for treatment effect bias from misclassification of responder status on PRO measures can lead to more trustworthy interpretation and effective decision-making.Clinicaltrials.gov: NCT00343200

Comparison of censoring assumptions to reduce bias in tuberculosis treatment cohort analyses.

ObjectiveObservational tuberculosis cohort studies are often limited by a lack of long-term data characterizing survival beyond the initial treatment outcome. Though Cox proportional hazards models are often applied to these data, differential risk of long-term survival, dependent on the initial treatment outcome, can lead to violations of model assumptions. We evaluate the performance of two alternate censoring approaches on reducing bias in treatment effect estimates.DesignWe simulate a typical multidrug-resistant tuberculosis cohort study and use Cox proportional hazards models to assess the relationship of an aggressive treatment regimen with hazard of death. We compare three assumptions regarding censored observations to determine which produces least biased treatment effect estimates: conventional non-informative censoring, an extension of short-term survival informed by literature, and incorporation of predicted long-term vital status.ResultsThe treatment regimen’s protective effect on death is consistently underestimated by the conventional censoring method, up to 7.6%. Models using the two alternative censoring techniques produce treatment effect estimates consistently stronger and less biased than the conventional method, underestimating the treatment effect by less than 2.4% across all scenarios.ConclusionsWhen model assumptions are violated, alternative censoring techniques can more accurately estimate associations between treatment and long-term survival. In multidrug-resistant tuberculosis cohort analyses, this bias reduction may yield more accurate and, larger effect estimates. This bias reduction can be achieved through use of standard statistical procedures with a simple re-coding of the censoring indicator.

Simulation Study to Estimate Bias of Treatment Effect in Pain Scores in Diabetic Neuropathy

Simulation Study to Estimate Bias of Treatment Effect in Pain Scores in Diabetic Neuropathy

PMU87 MATCHING ADJUSTED INDIRECT COMPARISONS: RECOMMENDATION FOR THE SELECTION OF BASELINE CHARACTERISTICS USED FOR MATCHING

Matching-adjusted indirect comparisons (MAICs) offer an alternative in situations where Bucher comparisons are likely to be biased due to differences in trial populations. MAIC applies weights to the individual patient data (IPD) from one trial, matching a set of baseline characteristics to those in the other trial(s). The selection of the baseline characteristics used for adjustment can impact the reliability of an MAIC. We propose a tool to assist in the selection of baseline characteristics used for adjustment in an MAIC. When deciding whether to include a baseline characteristic, the tool trades off its impact on effective sample size and its impact on the bias of treatment effects. The metric provided by the tool can be used to rank potential sets of baseline characteristics used for matching. We apply this tool to an anchored MAIC comparing the active treatments in two randomized controlled trials. Starting from a recommended set of baseline characteristics to be used for adjustment, the tool included any variables that did not reduce the effective sample size substantially, and excluded variables that did not change the weighted outcome substantially. Using these heuristics led to adequate effective sample size and did not introduce unnecessary uncertainty without substantially impacting estimated treatment effects. For example, excluding a disease severity index from the adjustment had less than 4% impact on the effect size but reduced the standard error of the outcome by about 30%. A set of baseline characteristics without this disease severity index was ranked higher in the recommendation than the set including the disease severity index, when other characteristics in the two sets were the same. The analyses show that improvements to standard error may be possible without an increase in bias, and suggest that statistical tools to identify such situations may be valuable.

Use of predicted vital status to improve survival analysis of multidrug-resistant tuberculosis cohorts

BackgroundMultidrug-resistant tuberculosis (MDR-TB) cohorts often lack long-term survival data, and are summarized instead by initial treatment outcomes. When using Cox proportional hazards models to analyze these cohorts, this leads to censoring subjects at the time of the initial treatment outcome, instead of them providing full survival data. This may violate the non-informative censoring assumption of the model and may produce biased effect estimates. To address this problem, we develop a tool to predict vital status at the end of a cohort period using the initial treatment outcome and assess its ability to reduce bias in treatment effect estimates.MethodsWe derive and apply a logistic regression model to predict vital status at the end of the cohort period and modify the unobserved survival outcomes to better match the predicted survival experience of study subjects. We compare hazard ratio estimates for effect of an aggressive treatment regimen from Cox proportional hazards models using time to initial treatment outcome, predicted vital status, and true vital status at the end of the cohort period.ResultsModels fit from initial treatment outcomes underestimate treatment effects by up to 22.1%, while using predicted vital status reduced this bias by 5.4%. Models utilizing the predicted vital status produce effect estimates consistently stronger and closer to the true treatment effect than estimates produced by models using the initial treatment outcome.ConclusionsAlthough studies often use initial treatment outcomes to estimate treatment effects, this may violate the non-informative censoring assumption of the Cox proportional hazards model and result in biased treatment effect estimates. Using predicted vital status at the end of the cohort period may reduce this bias in the analyses of MDR-TB treatment cohorts, yielding more accurate, and likely larger, treatment effect estimates. Further, these larger effect sizes can have downstream impacts on future study design by increasing power and reducing sample size needs.

Impact of flexible assessment schedules on treatment effect bias in pragmatic clinical trials.

e14519Background: Traditional clinical trials in Oncology require radiology assessments to be performed within a rigid timeframe that is burdensome and not indicative of real world practice. Pragma...

Dealing with competing risks in clinical trials: How to choose the primary efficacy analysis?

We investigate different primary efficacy analysis approaches for a 2-armed randomized clinical trial when interest is focused on a time to event primary outcome that is subject to a competing risk. We extend the work of Friedlin and Korn (2005) by considering estimation as well as testing and by simulating the primary and competing events' times from both a cause-specific hazards model as well as a joint subdistribution-cause-specific hazards model. We show that the cumulative incidence function can provide useful prognostic information for a particular patient but is not advisable for the primary efficacy analysis. Instead, it is preferable to fit a Cox model for the primary event which treats the competing event as an independent censoring. This is reasonably robust for controlling type I error and treatment effect bias with respect to the true primary and competing events' cause-specific hazards model, even when there is a shared, moderately prognostic, unobserved baseline frailty for the primary and competing events in that model. However, when it is plausible that a strongly prognostic frailty exists, combining the primary and competing events into a composite event should be considered. Finally, when there is an a priori interest in having both the primary and competing events in the primary analysis, we compare a bivariate approach for establishing overall treatment efficacy to the composite event approach. The ideas are illustrated by analyzing the Women's Health Initiative clinical trials sponsored by the National Heart, Lung, and Blood Institute.

The Effect of Workforce Mobility on Intervention Effectiveness Estimates.

Little is known about how mobile populations of workers may influence the ability to implement, measure, and evaluate health and safety interventions delivered at worksites. A simulation study is used to objectively measure both precision and relative bias of six different analytic methods as a function of the amount of mobility observed in the workforce. Those six methods are then used to reanalyze a previously conducted cluster-randomized control trial involving a highly mobile workforce in the construction industry. As workforce mobility increases, relative bias in treatment effects derived from standard models to analyze cluster-randomized trials also increases. Controlling for amount of time exposed to the intervention can greatly reduce this bias. Analyzing only subsets of workers who exhibit the least amount of mobility can result in decreased precision of treatment effect estimates. We demonstrate a 59% increase in the treatment effect size from the reanalysis of the previously conducted trial. When evaluating organizational interventions implemented at specific worksites by measuring perceptions and outcomes of workers present at those sites, researchers should consider the effects that the mobility of the workforce may have on the estimated treatment effects. The choice of analytic method can greatly affect both precision and accuracy of estimates.

Generalizability analysis for clinical trials: a simulation study.

Subjects are rarely selected on a random basis from a well-defined patient population of interest into a clinical trial, with women, children, the elderly, and those with common comorbidities who are frequently underrepresented. Decades of clinical experience have demonstrated that the application of trial findings to individual patients is permissible by using efficacy as a measure of effectiveness and assuming that the characteristics of patients are sufficiently similar. In order to investigate this issue in greater depth, we simulated a patient population with treatment effect size of 0.5 (Cohen's d) and five covariates that included gender, health insurance, comorbidity, age, and motivation. To demonstrate how selection of patients for a clinical trial can bias the results when treatment effect varies across individuals, we created 50 nonrandom clinical trials based on this patient population and showed relative bias to range from 1.68% to 99.70%. We calculated and evaluated three indexes: C-statistics, standardized mean difference (SMD), and Tipton's index (β) of generalization for the 50 nonrandom trials. Findings indicated that (i) the ranges were 0.56-0.98, 0.23-11.17, and 0.99-0.73 for C-statistics, SMD, and β, respectively, when treatment effect bias increased from 1.68% to 99.70% and (ii) C-statistics < 0.86, SMD < 1.95, and β > 0.91 when treatment effect bias <50%. Recommendations are made using existing generalization indexes on the basis of our simulation results. An example from a real clinical trial is provided for illustration. Copyright © 2017 John Wiley & Sons, Ltd.

Analysis of an incomplete longitudinal composite variable using a marginalized random effects model and multiple imputation.

The total score, which is calculated as the sum of scores in multiple items or questions, is repeatedly measured in longitudinal clinical studies. A mixed effects model for repeated measures method is often used to analyze these data; however, if one or more individual items are not measured, the method cannot be directly applied to the total score. We develop two simple and interpretable procedures that infer fixed effects for a longitudinal continuous composite variable. These procedures consider that the items that compose the total score are multivariate longitudinal continuous data and, simultaneously, handle subject-level and item-level missing data. One procedure is based on a multivariate marginalized random effects model with a multiple of Kronecker product covariance matrices for serial time dependence and correlation among items. The other procedure is based on a multiple imputation approach with a multivariate normal model. In terms of the type-1 error rate and the bias of treatment effect in total score, the marginalized random effects model and multiple imputation procedures performed better than the standard mixed effects model for repeated measures analysis with listwise deletion and single imputations for handling item-level missing data. In particular, the mixed effects model for repeated measures with listwise deletion resulted in substantial inflation of the type-1 error rate. The marginalized random effects model and multiple imputation methods provide for a more efficient analysis by fully utilizing the partially available data, compared to the mixed effects model for repeated measures method with listwise deletion.

Self-monitoring of lower leg skin temperature: accuracy of self-reported data and adherence to a cooling protocol for the prevention of venous leg ulcers.

BackgroundFor intervention studies that require the use of participant self-reports, the quality and accuracy of recorded data and variability in participant adherence rates to the treatment can cause significant outcome bias.PurposeTo assess the quality and accuracy of participant documentation of daily self-monitoring of leg skin temperature, adherence to a graduated cooling treatment protocol to prevent venous leg ulcers, and the potential for bias in treatment effect in a randomized controlled trial that included a population with chronic venous disease.MethodsIndividuals were randomized to a leg cooling intervention or placebo treatment group to daily self-monitor and record lower leg skin temperature over a 9-month period on monthly paper study logs. Returned study logs for the first 100 completed participants (n=54 cooling intervention, n=46 control) were reviewed for quality and accuracy. Adherence was determined from evaluating the accuracy of participant documentation. To examine potential outcome bias in treatment effect, mean between group and within group comparisons of the before and after treatment differences were conducted using an intention-to-treat (ITT) versus a modified intention-to-treat (mITT) analysis approach with an 85% accuracy cut-off rate. Data were collected in 2011–2014.ResultsOf the expected 900 study logs, 91.8% (826/900) were returned and 8.2% (74/900) were not. Non-mutually exclusive main error types in returned documentation included: 59.2% (489/826) white-outs, cross-off and/or overwrites, 34.9% (288/826) entries omitted, 29.4% (243/826) no performance of daily self-monitoring, 28.7% (237/826) no performance of the treatment intervention per the prescribed protocol regime, 26.8% (221/826) extraneous data, 8.6% (71/826) suspected fabrication, and 7.6% (63/826) questionable validity. Under ITT analysis, 38.4% (346/900) of all returned logs were <85% accurate, 25.0% (225/900) were 85%–99% accurate, and 36.6% (329/900) were 100% accurate. Mean overall participant adherence rates were: 22.0% at <85% accuracy, 53.0% at 85%–99% accuracy, and 25.0% at 100% accuracy. Under the mITT analysis, 54.0% (483/900) of returned logs were deemed adherent with ≥85% accuracy.ConclusionThis study found good rates of adherence. Under ITT analysis, 78.0% of participants were deemed adherent to the study protocol with ≥85% accuracy in documenting daily self-monitoring of skin temperatures in response to a topically applied experimental cooling cuff intervention for the prevention of venous leg ulcers.

Errors in self-reports of health services use: impact on alzheimer disease clinical trial designs.

Most Alzheimer disease clinical trials that compare the use of health services rely on reports of caregivers. The goal of this study was to assess the accuracy of self-reports among older adults with Alzheimer disease and their caregiver proxy respondents. This issue is particularly relevant to Alzheimer disease clinical trials because inaccuracy can lead both to loss of power and increased bias in study outcomes. We compared respondent accuracy in reporting any use and in reporting the frequency of use with actual utilization data as documented in a comprehensive database. We next simulated the impact of underreporting and overreporting on sample size estimates and treatment effect bias for clinical trials comparing utilization between experimental groups. Respondents self-reports have a poor level of accuracy with κ-values often below 0.5. Respondents tend to underreport use even for rare events such as hospitalizations and nursing home stays. In analyses simulating underreporting and overreporting of varying magnitude, we found that errors in self-reports can increase the required sample size by 15% to 30%. In addition, bias in the reported treatment effect ranged from 3% to 18% due to both underreporting and overreporting errors. Use of self-report data in clinical trials of Alzheimer disease treatments may inflate sample size needs. Even when adequate power is achieved by increasing sample size, reporting errors can result in a biased estimate of the true effect size of the intervention.

Intention-to-treat analyses and missing data approaches in pharmacotherapy trials for alcohol use disorders

ObjectivesIntention to treat (ITT) is an analytic strategy for reducing potential bias in treatment effects arising from missing data in randomised controlled trials (RCTs). Currently, no universally accepted definition of...

Optimizing Drug Development Programs: Type 2 Diabetes Case Study.

Recently, consideration was given to the impact of dose selection strategies in phase IIb on the overall success of drug development programs. A natural next step is to simultaneously optimize design aspects of both phase IIB and phase III. We used type 2 diabetes as an example, including realistic regulatory and commercial scenarios for this indication. The expected net present value (eNPV) has been selected as the primary outcome because it naturally accommodates optimization, providing an explicit trade-off between the probability of success (PoS) and time delays and trial costs. Our findings are that larger studies and/or implementation of an adaptive design over a fixed design in phase IIb provide more precise dose selection and reduce the bias of treatment effects and uncertainty in the estimated eNPV within the range of sample sizes that we examined. Developers also have to ensure that dose selection criteria are consistent with development strategy and objectives.