Competing Risk Bias Research Articles

Methodological biases in observational hospital studies of COVID-19 treatment effectiveness: pitfalls and potential.

This study aims to discuss and assess the impact of three prevalent methodological biases: competing risks, immortal-time bias, and confounding bias in real-world observational studies evaluating treatment effectiveness. We use a demonstrative observational data example of COVID-19 patients to assess the impact of these biases and propose potential solutions. We describe competing risks, immortal-time bias, and time-fixed confounding bias by evaluating treatment effectiveness in hospitalized patients with COVID-19. For our demonstrative analysis, we use observational data from the registry of patients with COVID-19 who were admitted to the Bellvitge University Hospital in Spain from March 2020 to February 2021 and met our predefined inclusion criteria. We compare estimates of a single-dose, time-dependent treatment with the standard of care. We analyze the treatment effectiveness using common statistical approaches, either by ignoring or only partially accounting for the methodological biases. To address these challenges, we emulate a target trial through the clone-censor-weight approach. Overlooking competing risk bias and employing the naïve Kaplan-Meier estimator led to increased in-hospital death probabilities in patients with COVID-19. Specifically, in the treatment effectiveness analysis, the Kaplan-Meier estimator resulted in an in-hospital mortality of 45.6% for treated patients and 59.0% for untreated patients. In contrast, employing an emulated trial framework with the weighted Aalen-Johansen estimator, we observed that in-hospital death probabilities were reduced to 27.9% in the "X"-treated arm and 40.1% in the non-"X"-treated arm. Immortal-time bias led to an underestimated hazard ratio of treatment. Overlooking competing risks, immortal-time bias, and confounding bias leads to shifted estimates of treatment effects. Applying the naïve Kaplan-Meier method resulted in the most biased results and overestimated probabilities for the primary outcome in analyses of hospital data from COVID-19 patients. This overestimation could mislead clinical decision-making. Both immortal-time bias and confounding bias must be addressed in assessments of treatment effectiveness. The trial emulation framework offers a potential solution to address all three methodological biases.

The population-attributable fraction for time-to-event data.

Even though the population-attributable fraction (PAF) is a well-established metric, it is often incorrectly estimated or interpreted not only in clinical application, but also in statistical research articles. The risk of bias is especially high in more complex time-to-event data settings. We explain how the PAF can be defined, identified and estimated in time-to-event settings with competing risks and time-dependent exposures. By using multi-state methodology and inverse probability weighting, we demonstrate how to reduce or completely avoid severe types of biases including competing risks bias, immortal time bias and confounding due to both baseline and time-varying patient characteristics. The method is exemplarily applied to a real data set. Moreover, we estimate the number of deaths that were attributable to ventilator-associated pneumonia in France in the year 2016. The example demonstrates how, under certain simplifying assumptions, PAF estimates can be extrapolated to a target population of interest. Defining and estimating the PAF in advanced time-to-event settings within a framework that unifies causal and multi-state modelling enables to tackle common sources of bias and allows straightforward implementation with standard software packages.

Long-term relative survival in uveal melanoma: a systematic review and meta-analysis

BackgroundA large proportion of patients with uveal melanoma develop metastases and succumb to their disease. Reports on the size of this proportion vary considerably.MethodsPubMed, Web of Science and Embase were searched for articles published after 1980. Studies with ≥100 patients reporting ≥five-year relative survival rates were included. Studies solely reporting Kaplan-Meier estimates and cumulative incidences were not considered, due to risk for competing risk bias and classification errors. A meta-analysis was performed using random-effects and weighted averages models, as well as a combined estimate based on curve fitting.ResultsNine studies and a total of 18 495 patients are included. Overall, the risk of selective reporting bias is low. Relative survival rates vary across the population of studies (I2 48 to 97% and Qp < 0.00001 to 0.15), likely due to differences in baseline characteristics and the large number of patients included (τ2 < 0.02). The 30-year relative survival rates follow a cubic curve that is well fitted to data from the random-effects inverse-variance and weighted average models (R2 = 0.95, p = 7.19E−7). The estimated five, ten, 15, 20, 25 and 30-year relative survival rates are 79, 66, 60, 60, 62 and 67%, respectively.ConclusionsThe findings suggest that about two in five of all patients with uveal melanoma ultimately succumb to their disease. This indicates a slightly better prognosis than what is often assumed, and that patients surviving 20 years or longer may have a survival advantage to individuals of the same sex and age from the general population.

Competing Risk Bias in Prognostic Models Predicting Hepatocellular Carcinoma Occurrence: Impact on Clinical Decision-making.

Existing models predicting hepatocellular carcinoma (HCC) occurrence do not account for competing risk events and, thus, may overestimate the probability of HCC. Our goal was to quantify this bias for patients with cirrhosis and cured hepatitis C. We analyzed a nationwide cohort of patients with cirrhosis and cured hepatitis C infection from Scotland. Two HCC prognostic models were developed: (1) a Cox regression model ignoring competing risk events and (2) a Fine-Gray regression model accounting for non-HCC mortality as a competing risk. Both models included the same set of prognostic factors used by previously developed HCC prognostic models. Two predictions were calculated for each patient: first, the 3-year probability of HCC predicted by model 1 and second, the 3-year probability of HCC predicted by model2. The study population comprised 1629 patients with cirrhosis and cured HCV, followed for 3.8 years on average. A total of 82 incident HCC events and 159 competing risk events (ie, non-HCC deaths) were observed. The mean predicted 3-year probability of HCC was 3.37% for model 1 (Cox) and 3.24% for model 2 (Fine-Gray). For most patients (76%), the difference in the 3-year probability of HCC predicted by model 1 and model 2 was minimal (ie, within 0 to ±0.3%). A total of 2.6% of patients had a large discrepancy exceeding 2%; however, these were all patients with a 3-year probability exceeding >5% in both models. Prognostic models that ignore competing risks do overestimate the future probability of developing HCC. However, the degree of overestimation-and the way it is patterned-means that the impact on HCC screening decisions is likely to be modest.

Response to "Overlooked Shortcomings of Observational Studies of Interventions in Coronavirus Disease 2019: An Illustrated Review for the Clinician" by Tleyjeh et al.

Abstract We commented on the publication by Tleyjeh et al regarding the overlooked shortcomings of observational studies of interventions in Coronavirus Disease 2019. Although we agree with Tleyjeh and colleagues on the issue of the competing risk bias in observational studies, the recommendations on the application of the Fine-Gray model provided by the authors are incomplete. The Fine-Gray approach may not be suitable in the presence of interval time-dependent covariates, that are often the case in the studies assessing therapeutic interventions for patients with Coronavirus Disease 2019.

Association Between Alzheimer Disease and Cancer With Evaluation of Study Biases

Observational studies consistently report inverse associations between cancer and Alzheimer disease (AD). Shared inverse etiological mechanisms might explain this phenomenon, but a systematic evaluation of methodological biases in existing studies is needed. To systematically review and meta-analyze evidence on the association between cancer and subsequent AD, systematically identify potential methodological biases in studies, and estimate the influence of these biases on the estimated pooled association between cancer and AD. All-language publications were identified from PubMed, Embase, and PsycINFO databases through September 2, 2020. Longitudinal cohort studies and case-control studies on the risk of AD in older adults with a history of any cancer type, prostate cancer, breast cancer, colorectal cancer, or nonmelanoma skin cancer, relative to those with no cancer history. Two reviewers independently abstracted the data and evaluated study biases related to confounding, diagnostic bias, competing risks, or survival bias. Random-effects meta-analysis was used to provide pooled estimates of the association between cancer and AD. Metaregressions were used to evaluate whether the observed pooled estimate could be attributable to each bias. The study was designed and conducted according to the Preferring Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) reporting guideline. Incidence, hazard, or odds ratios for AD comparing older adults with vs without a previous cancer diagnosis. In total, 19 cohort studies and 3 case-control studies of the associations between any cancer type (n = 13), prostate cancer (n = 5), breast cancer (n = 1), and nonmelanoma skin cancer (n = 3) with AD were identified, representing 9 630 435 individuals. In all studies combined, cancer was associated with decreased AD incidence (cohort studies: random-effects hazard ratio, 0.89; 95% CI, 0.79-1.00; case-control studies: random-effects odds ratio, 0.75; 95% CI, 0.61-0.93). Studies with insufficient or inappropriate confounder control or greater likelihood of AD diagnostic bias had mean hazard ratios closer to the null value, indicating that these biases could not explain the observed inverse association. Competing risks bias was rare. Studies with greater likelihood of survival bias had mean hazard ratios farther from the null value. The weak inverse association between cancer and AD may reflect shared inverse etiological mechanisms or survival bias but is not likely attributable to diagnostic bias, competing risks bias, or insufficient or inappropriate control for potential confounding factors.

Why we should take care of the competing risk bias in survival analysis: A phase II trial on the toxicity profile of radiotherapy for prostate cancer.

The aim of the present study is to evaluate and quantify the bias of competing risks in an Italian oncologic cohort comparing results from different statistical analysis methods. Competing risks are very common in randomized clinical trials and observational studies, in particular oncology and radiotherapy ones, and their inappropriate management causes results distortions widely present in clinical scientific articles. This is a single-institution phase II trial including 41 patients affected by prostate cancer and undergoing radiotherapy (IMRT-SIB) at the University Hospital of Udine.Different outcomes were considered: late toxicities, relapse, death.Death in the absence of relapse or late toxicity was considered as a competing event. The Kaplan Meier method, compared to cumulative incidence function method, overestimated the probability of the event of interest (toxicity and biochemical relapse) and of the competing event (death without toxicity/relapse) by 9.36%. The log-rank test, compared to Gray's test, overestimated the probability of the event of interest by 5.26%.The Hazard Ratio's and cause specific hazard's Cox regression are not directly comparable to subdistribution hazard's Fine and Gray's modified Cox regression; nonetheless, the FG model, the best choice for prognostic studies with competing risks, found significant associations not emerging with Cox regression. This study confirms that using inappropriate statistical methods produces a 10% overestimation in results, as described in the literature, and highlights the importance of taking into account the competing risks bias.

Influence of competing risks on estimates of recurrence risk and breast cancer-specific mortality in analyses of the Early Breast Cancer Trialists Collaborative Group.

533 Background: Early stage breast cancer is a curable disease with the majority of patients dying of causes other than breast cancer. The influence of these competing risks of death on the interpretation of Kaplan-Meier(KM)-based analyses such as those performed by the Early Breast Cancer Trialists Collaborative Group (EBCTCG) are unknown. Methods: We searched the Clinical Trial Service Unit and Epidemiological Studies Unit website at Oxford University to identify all meta-analyses published by the EBCTCG between 2005 and 2018. Studies were included if they contained KM curves with risk estimates for either breast cancer mortality and/or breast cancer recurrence. The potential influence of competing risks was estimated using a validated multivariate linear model that predicts the amount that the KM risk estimates are biased relative to outcome risk measured with the cumulative incidence function (CIF). Results: The initial search identified 14 analyses published by the EBCTCG with 10 of the 14 studies (71%) susceptible to competing risk bias cited both the number of events of interest and competing events. Eight of the ten studies (80%) had a relative difference between the KM estimate and the competing risk adjusted estimate of more than 10% while 2 of 10 (20%) had a difference of less than 10%. The relative difference between the KM and adjusted estimates was 28.4% for local recurrence, 16.8% for distant recurrence, and 6.7% for breast cancer-specific mortality. There was 2.2% difference between KM and adjusted analyses between 0-4 years and 18.9% beyond 10 years of follow up. Use of KM and CIF-based analysis did not influence treatment effect in the majority of included studies. Conclusions: This study provides estimates for the overestimation of risk in Kaplan-Meier analyses resulting from failure to address competing risk bias. CIFs are more appropriate to measure outcome risk over time and should be used especially for long-term follow-up studies and for analysis of rare events.

Accuracy of massive transfusion as a surrogate for significant traumatic bleeding in health administrative datasets

Background: Due to the challenge of identifying need for intervention in bleeding patients, there is a growing interest in prediction modeling. Massive transfusion (MT; 10 or more packed red cells in 24 h) is the most commonly studieddependent variable, serving as a surrogate for severe bleeding and its prediction guides the need for intervention. The critical administration threshold (CAT; 3 packed red cells in 1 h) has been proposed as an alternative. In this study, we aim to compare the classification accuracy of these two surrogates for hemorrhage-related outcomes in health administrative datasets.Methods: We performed a secondary analysis of major trauma patients from the prospectively collected Ottawa Trauma Registry, from September 2014 to September 2017. We conducted a logistic regression analysis utilizing need for hemostasis or hemorrhagic death as dependent variables. We compared classification accuracy in terms of sensitivity, specificity, positive predictive value, negative predictive value and AUC. CAT + and MT + status is not mutually exclusive.Results: We studied 890 major trauma patients, including 145 CAT + and 48 MT + patients. CAT + demonstrated a superior association for the composite outcome of 24-hour hemorrhage-related mortality and need for hemostasis (AUC 0.815 vs. 0.644, p < 0.0001). This performance was driven by a substantial difference in sensitivity, noted to be 70.0% (95% CI 62.1–77.9%) for CAT + but only 30.0% (95% CI 22.1–37.9%) for MT+. CAT + and MT + demonstrated specificities of 92.9% (95% CI 91.1–94.7%) and 98.9% (98.1–99.6%) respectively.Conclusion: This study illustrates the concepts of survivorship and competing risk bias for massive transfusion. Utilizing a composite outcome of need for hemostasis and early hemorrhagic death, we demonstrate that CAT + is more accurate for identifying significantly bleeding patients.

Improving nested case-control studies to conduct a full competing-risks analysis for nosocomial infections.

Competing risks are a necessary consideration when analyzing risk factors for nosocomial infections (NIs). In this article, we identify additional information that a competing risks analysis provides in a hospital setting. Furthermore, we improve on established methods for nested case-control designs to acquire this information. Using data from 2 Spanish intensive care units and model simulations, we show how controls selected by time-dynamic sampling for NI can be weighted to perform risk-factor analysis for death or discharge without infection. This extension not only enables hazard rate analysis for the competing risk, it also enables prediction analysis for NI. The estimates acquired from the extension were in good agreement with the results from the full (real and simulated) cohort dataset. The reduced dataset results averted any false interpretation common in a competing-risks setting. Using additional information that is routinely collected in a hospital setting, a nested case-control design can be successfully adapted to avoid a competing risks bias. Furthermore, this adapted method can be used to reanalyze past nested case-control studies to enhance their findings.

Survival biases lead to flawed conclusions in observational treatment studies of influenza patients

Background and ObjectiveSeveral observational studies reported that Oseltamivir (Tamiflu) reduced mortality in infected and hospitalized patients. Because of the restriction of observation to hospital stay and time-dependent treatment assignment, such findings were prone to common types of survival bias (length, time-dependent and competing risk bias). MethodsBritish hospital data from the Influenza Clinical Information Network (FLU-CIN) study group were used which included 1,391 patients with confirmed pandemic influenza A/H1N1 2009 infection. We used a multistate model approach with following states: hospital admission, Oseltamivir treatment, discharge, and death. Time origin is influenza onset. We displayed individual data, risk sets, hazards, and probabilities from multistate models to study the impact of these three common survival biases. ResultsThe correct hazard ratio of Oseltamivir for death was 1.03 (95% confidence interval [CI]: 0.64–1.66) and for discharge 1.89 (95% CI: 1.65–2.16). Length bias increased both hazard ratios (HRs): HR (death) = 1.82 (95% CI: 1.12–2.98) and HR (discharge) = 4.44 (95% CI: 3.90–5.05), whereas the time-dependent bias reduced them: HR (death) = 0.62 (95% CI: 0.39–1.00) and HR (discharge) = 0.85 (95% CI: 0.75–0.97). Length and time-dependent bias were less pronounced in terms of probabilities. Ignoring discharge as a competing event for hospital death led to a remarkable overestimation of hospital mortality and failed to detect the reducing effect of Oseltamivir on hospital stay. ConclusionsThe impact of each of the three survival biases was remarkable, and it can make neuraminidase inhibitors appear more effective or even harmful. Incorrect and misclassified risk sets were the primary sources of biased hazard rates.

Competing risk bias in Kaplan–Meier risk estimates can be corrected

BackgroundKaplan–Meier (KM) analyses are frequently used to measure outcome risk over time. These analyses overestimate risk whenever competing events are present. Many published KM analyses are susceptible to such competing risk bias. This study derived and validated a model that predicted true outcome risk based on the biased KM risk. MethodsWe simulated survival data sets having a broad range of 1-year true outcome and competing event risk. Unbiased true outcome risk estimates were calculated using the cumulative incidence function (CIF). Multiple linear regression was used to determine the independent association of CIF-based true outcome risk with the biased KM risk and the proportion of all outcomes that were competing events. ResultsThe final model found that both the biased KM-based risk and the proportion of all outcomes that were competing events were strongly associated with CIF-based risk. In validation populations that used a variety of distinct survival hazard functions, the model accurately predicted the CIF (R2 = 1). ConclusionsTrue outcome risk can be accurately predicted from KM estimates susceptible to competing risk bias.

Competing risk bias was common in Kaplan–Meier risk estimates published in prominent medical journals

ObjectiveRisk estimates from Kaplan–Meier curves are well known to medical researchers, reviewers, and editors. In this study, we determined the proportion of Kaplan–Meier analyses published in prominent medical journals that are potentially biased because of competing events (“competing risk bias”). Study Design and SettingWe randomly selected 100 studies that had at least one Kaplan–Meier analysis and were recently published in prominent medical journals. Susceptibility to competing risk bias was determined by examining the outcome and potential competing events. In susceptible studies, bias was quantified using a previously validated prediction model when the number of outcomes and competing events were given. ResultsForty-six studies (46%) contained Kaplan–Meier analyses susceptible to competing risk bias. Sixteen studies (34.8%) susceptible to competing risk cited the number of outcomes and competing events; in six of these studies (6/16, 37.5%), the outcome risk from the Kaplan–Meier estimate (relative to the true risk) was biased upward by 10% or more. ConclusionAlmost half of Kaplan–Meier analyses published in medical journals are susceptible to competing risk bias and may overestimate event risk. This bias was found to be quantitatively important in a third of such studies.

Interval censoring and competing risks when reporting results of glaucoma surgery

Sir, Dr Dulku1 criticised the Kaplan–Meier analysis that Drs Anand and Wechsler2 used to assess failure and complications after deep sclerectomy with mitomycin C in eyes with failed glaucoma surgery, pseudophakia, or both. He pointed out that these events had occurred at unknown times before the visit at which they were recorded, making the survival curves too good, and recommended that interval censoring3 adjust for this bias. However, competing risk bias should additionally be considered. Drs Anand and Wechsler operated on 82 patients,2 who were on average 76 years old. A total of 20 patients died during the over 5-year-long observation period.2 The authors do not mention how they dealt statistically with patients who died.2 We dare expect they were censored just like the patients who became too ill to attend their clinic.2 However, a fundamental difference exists between these two groups: only the latter group of patients remained at risk after censoring. After censoring, the Kaplan–Meier curve will drop proportionately more with any subsequent event as compared with what it would have dropped had censoring not taken place. A key assumption is that censoring is independent of the risk of experiencing the event of interest, that is, the risk is equal before and after censoring.4 Clearly, this assumption is not met if any subjects die: the survival curve will become too pessimistic. Death is a competing risk event, which should be dealt with methods other than censoring,5, 6 such as cumulative incidence analysis,7 found both in the R package mentioned and in a number of commercial packages. The paper of Drs Zhang and Sun,3 which Dr Dulku1 cites, briefly discusses interval censoring in face of competing risks. However, commercial software for this purpose is not yet marketed. Formal adjustment is available either for interval censoring1, 3, 6 or for competing risks.5, 7, 8, 9, 10 To address simultaneously both biases, one reasonable approach at present is to undertake cumulative incidence analysis and to plot two curves, the first modelling the event of interest as occurring when it was recorded, and the second assigning it to the immediately preceding visit. The former curve will exaggerate the probability of success and the latter the probability of failure. Alternatively, a cumulative incidence curve based on the midpoint of the review interval may be used as an approximation of interval censoring.9 Interval censoring and competing risks bias in survival analysis are ill known to authors, reviewers, and readers, risking misinterpretation of study results.

Competing risk bias to explain the inverse relationship between smoking and malignant melanoma

The relationship between smoking and melanoma remains unclear. Among the different results is the paradoxical finding that smoking was shown to be inversely associated with the risk of malignant melanoma in some large cohort and case-control studies, even after control for suspected confounding variables. Smoking is a known risk factor for many non-communicable diseases, including coronary heart disease, stroke, as well as other malignancies; it has been shown to be positively associated with other types of skin cancer, and there remains no clear biologic explanation for a possible protective effect on malignant melanoma. In this paper, we propose a plausible mechanism of bias from smoking-related competing risks that may explain or contribute to the inverse association between smoking and melanoma as spurious. Using directed acyclic graphs for formalization and visualization of assumptions, and Monte Carlo simulation techniques, we demonstrate how published inverse associations might be compatible with selection bias resulting from uncontrolled or unmeasured common causes of competing outcomes of smoking-related diseases and malignant melanoma. We present results from various scenarios assuming a true null as well as a true positive association between smoking and malignant melanoma. Under a true null assumption, we find inverse associations due to the biasing mechanism to be compatible with published results in the literature, especially after the addition of unmeasured confounding variables. This study could be seen as offering a cautionary note in the interpretation of published smoking-melanoma findings.