Differential Misclassification Research Articles

Cost-benefit of outcome adjudication in nine randomised stroke trials.

Central adjudication of outcomes is common for randomised trials and should control for differential misclassification. However, few studies have estimated the cost of the adjudication process. We estimated the cost of adjudicating the primary outcome in nine randomised stroke trials (25,436 participants). The costs included adjudicators' time, direct payments to adjudicators, and co-ordinating centre costs (e.g. uploading cranial scans and general set-up costs). The number of events corrected after adjudication was our measure of benefit. We calculated cost per corrected event for each trial and in total. The primary outcome in all nine trials was either stroke or a composite that included stroke. In total, the adjudication process associated with this primary outcome cost in excess of £100,000 for a third of the trials (3/9). Mean cost per event corrected by adjudication was £2295.10 (SD: £1482.42). Central adjudication is a time-consuming and potentially costly process. These costs need to be considered when designing a trial and should be evaluated alongside the potential benefits adjudication brings to determine whether they outweigh this expense.

Probabilistic Quantification of Bias to Combine the Strengths of Population-Based Register Data and Clinical Cohorts-Studying Mortality in Osteoarthritis.

We propose combining population-based register data with a nested clinical cohort to correct misclassification and unmeasured confounding through probabilistic quantification of bias. We have illustrated this approach by estimating the association between knee osteoarthritis and mortality. We used the Swedish Population Register to include all persons resident in the Skåne region in 2008 and assessed whether they had osteoarthritis using data from the Skåne Healthcare Register. We studied mortality through year 2017 by estimating hazard ratios. We used data from the Malmö Osteoarthritis Study (MOA), a small cohort study from Skåne, to derive bias parameters for probabilistic quantification of bias, to correct the hazard ratio estimate for differential misclassification of the knee osteoarthritis diagnosis and confounding from unmeasured obesity. We included 292,000 persons in the Skåne population and 1,419 from the MOA study. The adjusted association of knee osteoarthritis with all-cause mortality in the MOA sample had a hazard ratio of 1.10 (95% confidence interval (CI): 0.80, 1.52) and was thus inconclusive. The naive association in the Skåne population had a hazard ratio of 0.95 (95% CI: 0.93, 0.98), while the bias-corrected estimate was 1.02 (95% CI: 0.59, 1.52), suggesting high uncertainty in bias correction. Combining population-based register data with clinical cohorts provides more information than using either data source separately.

Identifying novel associations in GWAS by hierarchical Bayesian latent variable detection of differentially misclassified phenotypes

BackgroundHeterogeneity in the definition and measurement of complex diseases in Genome-Wide Association Studies (GWAS) may lead to misdiagnoses and misclassification errors that can significantly impact discovery of disease loci. While well appreciated, almost all analyses of GWAS data consider reported disease phenotype values as is without accounting for potential misclassification.ResultsHere, we introduce Phenotype Latent variable Extraction of disease misdiagnosis (PheLEx), a GWAS analysis framework that learns and corrects misclassified phenotypes using structured genotype associations within a dataset. PheLEx consists of a hierarchical Bayesian latent variable model, where inference of differential misclassification is accomplished using filtered genotypes while implementing a full mixed model to account for population structure and genetic relatedness in study populations. Through simulations, we show that the PheLEx framework dramatically improves recovery of the correct disease state when considering realistic allele effect sizes compared to existing methodologies designed for Bayesian recovery of disease phenotypes. We also demonstrate the potential of PheLEx for extracting new potential loci from existing GWAS data by analyzing bipolar disorder and epilepsy phenotypes available from the UK Biobank. From the PheLEx analysis of these data, we identified new candidate disease loci not previously reported for these datasets that have value for supplemental hypothesis generation.ConclusionPheLEx shows promise in reanalyzing GWAS datasets to provide supplemental candidate loci that are ignored by traditional GWAS analysis methodologies.

Effect of Smoking on Breast Cancer by Adjusting for Smoking Misclassification Bias and Confounders Using a Probabilistic Bias Analysis Method.

PurposeThe aim of this study was to determine the association between smoking and breast cancer after adjusting for smoking misclassification bias and confounders.MethodsIn this case–control study, 1000 women with breast cancer and 1000 healthy controls were selected. Using a probabilistic bias analysis method, the association between smoking and breast cancer was adjusted for the bias resulting from misclassification of smoking secondary to self-reporting as well as a minimally sufficient adjustment set of confounders derived from a causal directed acyclic graph (cDAG). Population attributable fraction (PAF) for smoking was calculated using Miettinen’s formula.ResultsWhile the odds ratio (OR) from the conventional logistic regression model between smoking and breast cancer was 0.64 (95% CI: 0.36–1.13), the adjusted ORs from the probabilistic bias analysis were in the ranges of 2.63–2.69 and 1.73–2.83 for non-differential and differential misclassification, respectively. PAF ranges obtained were 1.36–1.72% and 0.62–2.01% using the non-differential bias analysis and differential bias analysis, respectively.ConclusionAfter misclassification correction for smoking, the non-significant negative-adjusted association between smoking and breast cancer changed to a significant positive-adjusted association.

Are we missing opioid-related deaths among people with HIV?

BackgroundAscertainment of unnatural and overdose death may be unreliable among individuals with life-limiting conditions such as HIV infection. We sought to determine whether the relationship between opioid use and unnatural death differs among decedents with HIV (DWH) and those without. MethodsDecedents in the Veterans Aging Cohort Study (VACS) from 2002 to 14 were linked to the National Death Index cause of death file. Deaths were classified as unnatural, overdose (a subset of unnatural), or other. We defined opioid use as self-reported illicit use or receipt of prescribed opioids. Treating unnatural and overdose deaths as outcomes, we calculated odds ratios for opioid exposure by HIV status, with and without adjustment for disease severity using VACS Index. ResultsAmong 561 decedents without HIV (DWOH) and 884 DWH, 11 % and 8 % respectively were classified as unnatural deaths and 4 % and 2 % were classified as overdose deaths. Among DWOH, opioid use was associated with 2-fold greater odds of unnatural (OR 2.3; 95 % CI 1.3–4.0) and 4-fold greater odds of overdose death (OR 4.5; 95 % CI 1.5–13.7); in adjusted analyses, opioid use was associated with unnatural death (OR 2.6; 95 % CI 1.3–4.9) and with overdose (OR 4.2; 95 % CI 1.4–12.7). Opioid use was not associated with unnatural or overdose death among DWH. ConclusionOpioid use was strongly associated with unnatural and overdose death among DWOH but not among DWH suggesting potential differential misclassification. Caution should be used in interpreting prevalence, incidence and risk factors for unnatural and overdose cause of death among patients with life-limiting conditions such as HIV.

Nonparametric estimation of distributions and diagnostic accuracy based on group-tested results with differential misclassification.

This article concerns the problem of estimating a continuous distribution in a diseased or nondiseased population when only group-based test results on the disease status are available. The problem is challenging in that individual disease statuses are not observed and testing results are often subject to misclassification, with further complication that the misclassification may be differential as the group size and the number of the diseased individuals in the group vary. We propose a method to construct nonparametric estimation of the distribution and obtain its asymptotic properties. The performance of the distribution estimator is evaluated under various design considerations concerning group sizes and classification errors. The method is exemplified with data from the National Health and Nutrition Examination Survey study to estimate the distribution and diagnostic accuracy of C-reactive protein in blood samples in predicting chlamydiaincidence.

Outcome assessment by central adjudicators in randomised stroke trials: Simulation of differential and non-differential misclassification.

IntroductionAdjudication of the primary outcome in randomised trials is thought to control misclassification. We investigated the amount of misclassification needed before adjudication changed the primary trial results.Patients (or materials) and methods: We included data from five randomised stroke trials. Differential misclassification was introduced for each primary outcome until the estimated treatment effect was altered. This was simulated 1000 times. We calculated the between-simulation mean proportion of participants that needed to be differentially misclassified to alter the treatment effect. In addition, we simulated hypothetical trials with a binary outcome and varying sample size (1000–10,000), overall event rate (10%–50%) and treatment effect (0.67–0.90). We introduced non-differential misclassification until the treatment effect was non-significant at 5% level.ResultsFor the five trials, the range of unweighted kappa values were reduced from 0.89–0.97 to 0.65–0.85 before the treatment effect was altered. This corresponded to 2.1%–6% of participants misclassified differentially for trials with a binary outcome. For the hypothetical trials, those with a larger sample size, stronger treatment effect and overall event rate closer to 50% needed a higher proportion of events non-differentially misclassified before the treatment effect became non-significant.Discussion:We found that only a small amount of differential misclassification was required before adjudication altered the primary trial results, whereas a considerable proportion of participants needed to be misclassified non-differentially before adjudication changed trial conclusions. Given that differential misclassification should not occur in trials with sufficient blinding, these results suggest that central adjudication is of most use in studies with unblinded outcome assessment.Conclusion:For trials without adequate blinding, central adjudication is vital to control for differential misclassification. However, for large blinded trials, adjudication is of less importance and may not be necessary.

Direct Effects under Differential Misclassification in Outcomes, Exposures, and Mediators

AbstractDirect effects in mediation analysis quantify the effect of an exposure on an outcome not mediated by a certain intermediate. When estimating direct effects through measured data, misclassification may occur in the outcomes, exposures, and mediators. In mediation analysis, any such misclassification may lead to biased estimates in the direct effects. Basing on the conditional dependence between the mismeasured variable and other variables given the true variable, misclassification mechanisms can be divided into non-differential misclassification and differential misclassification. In this article, several scenarios of differential misclassification will be discussed and sensitivity analysis results on direct effects will be derived for those eligible scenarios. According to our findings, the estimated direct effects are not necessarily biased in intuitively predictable directions when the misclassification is differential. The bounds of the true effects are functions of measured effects and sensitivity parameters. An example from the 2018 NCHS data will illustrate how to conduct sensitivity analyses with our results on misclassified outcomes, gestational hypertension and eclampsia, when the exposure is Hispanic women versus non-Hispanic White women and the mediator is weights gain during pregnancy.

Incorporating retesting outcomes for estimation of disease prevalence.

Group testing has been widely used as a cost-effective strategy to screen for and estimate the prevalence of a rare disease. While it is well-recognized that retesting is necessary for identifying infected subjects, it is not required for estimating the prevalence. For a test without misclassification, gains in statistical efficiency are expected from incorporating retesting results in the estimation of the prevalence. However, when the test is subject to misclassification, it is not clear how much gain should be expected. There are a number of theoretical challenges in addressing this issue, including (1) enumerating the potential test results from retesting individual subjects in a group, (2) the dependence among these test results and the test result from testing at the group level, and (3) differential misclassification due to pooling of biospecimens. Overcoming some of these challenges, we show that retesting subjects in either positive or negative groups can substantially improve the efficiency of the estimation and that retesting positive groups yields higher efficiency than retesting a same number or proportion of negative groups.

MISCLASSIFICATION IN DEMENTIA DATA IN THE CARDIOVASCULAR HEALTH STUDY: A PROBABILISTIC BIAS ANALYSIS

Data routinely collected in observational studies from phone calls, medical records, and death certificates can be used to categorize dementia, though there may be misclassification. We applied probabilistic bias analyses to assess the magnitude, direction, and uncertainty of the error due to misclassification of dementia data in the Cardiovascular Health Study (CHS). We categorized dementia among all participants (73,284 person-years) using medications, ICD-9 codes, use of proxy, and death certificates, and compared to the gold standard adjudicated dementia in the CHS Memory Studies (28,250 person-years). Using the gold standard, positive (PPV) and negative predictive values (NPV) of dementia categorization were estimated within strata defined by age and sex. In probabilistic bias analyses, we reclassified participants from the full study using estimated PPVs and NPVs in 5,000 replicates. We estimated the hazard ratio (HR) of dementia associated with age, race, sex, hypertension, diabetes, and APOE4 genotype in this bias analysis and compared these results to those using original data. ICD-9 codes had low specificity and were excluded in further analyses. The NPV was differential by sex (66% for females and 79% for males) and race (51% for blacks, 60% for whites). In bias analysis, the HR for black race was attenuated from 2.81 (95%CI:1.36-5.80) to 1.23 (95%CI:1.14-1.33). The estimate for hypertension was statistically significant only in bias analysis. Estimates and inferences for the other covariates were modestly different. Differential misclassification may lead to important biases of risk factors, but can be recognized and addressed using probabilistic bias analyses.

An augmented estimation procedure for EHR-based association studies accounting for differential misclassification.

The ability to identify novel risk factors for health outcomes is a key strength of electronic health record (EHR)-based research. However, the validity of such studies is limited by error in EHR-derived phenotypes. The objective of this study was to develop a novel procedure for reducing bias in estimated associations between risk factors and phenotypes in EHR data. The proposed method combines the strengths of a gold-standard phenotype obtained through manual chart review for a small validation set of patients and an automatically-derived phenotype that is available for all patients but is potentially error-prone (hereafter referred to as the algorithm-derived phenotype). An augmented estimator of associations is obtained by optimally combining these 2 phenotypes. We conducted simulation studies to evaluate the performance of the augmented estimator and conducted an analysis of risk factors for second breast cancer events using data on a cohort from Kaiser Permanente Washington. The proposed method was shown to reduce bias relative to an estimator using only the algorithm-derived phenotype and reduce variance compared to an estimator using only the validation data. Our simulation studies and real data application demonstrate that, compared to the estimator using validation data only, the augmented estimator has lower variance (ie, higher statistical efficiency). Compared to the estimator using error-prone EHR-derived phenotypes, the augmented estimator has smaller bias. The proposed estimator can effectively combine an error-prone phenotype with gold-standard data from a limited chart review in order to improve analyses of risk factors using EHR data.

Classification of Medicaid Coverage on Birth Records in Wisconsin, 2011-2012.

In 2011, Wisconsin introduced the 2003 Revision of the US Standard Certificate of Live Birth, which includes a variable for principal payer. This variable could help in estimating Medicaid coverage for delivery services, but its accuracy in most states is not known. Our objective was to validate Medicaid payer classification on Wisconsin birth records. We linked 128 141 Wisconsin birth records (2011-2012 calendar years) to 54 600 Medicaid claims. Using claims as the gold standard, we measured the payer variable's validity (sensitivity, specificity, positive predictive value [PPV], negative predictive value [NPV]) overall and by maternal age, race/ethnicity, education, facility delivery volume, and the Medicaid proportion of facility delivery volume. Multivariable log-binomial regression tested the association between each characteristic and payer misclassification among Medicaid-paid births. Of 128 141 birth records, 50 652 (39.5%) indicated Medicaid as the principal payer and 54 600 (42.6%) linked to a Medicaid claim. The birth record misclassified 10 007 of 54 600 (18.3%) Medicaid-paid births as non-Medicaid and 6059 of 73 541 (8.2%) non-Medicaid births as Medicaid-paid. The payer variable was less sensitive (81.7%) than specific (91.8%), and PPV and NPV were similar (∼88%). Sensitivity was highest among mothers who were Hispanic, had no high school diploma, or delivered in Medicaid-majority delivery facilities. Maternal age ≥40, maternal education >high school, and delivering in a non-Medicaid-majority delivery facility were positively associated with payer misclassification among Medicaid-paid births. Differential misclassification of principal payer in the birth record may bias risk surveillance of Medicaid deliveries.

Adjusting for differential misclassification in matched case-control studies utilizing health administrative data.

In epidemiological studies of secondary data sources, lack of accurate disease classifications often requires investigators to rely on diagnostic codes generated by physicians or hospital systems to identify case and control groups, resulting in a less-than-perfect assessment of the disease under investigation. Moreover, because of differences in coding practices by physicians, it is hard to determine the factors that affect the chance of an incorrectly assigned disease status. What results is a dilemma where assumptions of non-differential misclassification are questionable but, at the same time, necessary to proceed with statistical analyses. This paper develops an approach to adjust exposure-disease association estimates for disease misclassification, without the need of simplifying non-differentiality assumptions, or prior information about a complicated classification mechanism. We propose to leverage rich temporal information on disease-specific healthcare utilization to estimate each participant's probability of being a true case and to use these estimates as weights in a Bayesian analysis of matched case-control data. The approach is applied to data from a recent observational study into the early symptoms of multiple sclerosis (MS), where MS cases were identified from Canadian health administrative databases and matched to population controls that are assumed to be correctly classified. A comparison of our results with those from non-differentially adjusted analyses reveals conflicting inferences and highlights that ill-suited assumptions of non-differential misclassification can exacerbate biases in association estimates.

Improving the reporting of cancer-specific mortality and survival in research using cancer registry data

Population-based registries are increasingly used in cancer research. In such studies, cancer-specific mortality or survival is frequently used as the primary outcome. To determine whether a putative cancer was part of the causal chain of events leading to death, cancer registries primarily rely on death certificates. Hence, they depend on the subjective interpretation of information available to medical examiners at the time of death. Misclassification may occur: studies report misclassification of cancer as a cause of death in 15%–35% of death certificates based on evaluation by expert panels and/or autopsy reports. Further misclassification may occur when coding death causes in the cancer registry. Researchers should be aware of potential misclassification bias when using cancer registry data. Differential misclassification may bias the results towards or away from the null hypothesis, depending on whether there is relative over- or under-reporting of cancer-related deaths in one group. Strategies to improve reporting of cancer-specific survival/mortality include (1) describing the procedure used to identify cancer-specific deaths; (2) considering the use of multiple definitions of cancer-related deaths (strict/liberal definitions of cancer-specific deaths, and/or addition of relative survival as an outcome); and (3) reporting cancer-specific survival/mortality together with the objectively measured parameters overall survival or all-cause mortality.

Inflation of type I error rates due to differential misclassification in EHR‐derived outcomes: Empirical illustration using breast cancer recurrence

Many outcomes derived from electronic health records (EHR) not only are imperfect but also may suffer from exposure-dependent differential misclassification due to variability in the quality and availability of EHR data across exposure groups. The objective of this study was to quantify the inflation of type I error rates that can result from differential outcome misclassification. We used data on gold-standard and EHR-derived second breast cancers in a cohort of women with a prior breast cancer diagnosis from 1993 to 2006 enrolled in Kaiser Permanente Washington. We simulated an exposure that was independent of the true outcome status. A surrogate outcome was then simulated with varying sensitivity and specificity according to exposure status. We estimated the type I error rate for a test of association relating this exposure to the surrogate outcome, while varying outcome sensitivity and specificity in exposed individuals. Type I error rates were substantially inflated above the nominal level (5%) for even modest departures from nondifferential misclassification. Holding sensitivity in exposed and unexposed groups at 85%, a difference in specificity of 10% between the exposed and unexposed (80% vs 90%) resulted in a 36% type I error rate. Type I error was inflated more by differential specificity than sensitivity. Differential outcome misclassification may induce spurious findings. Researchers using EHR-derived outcomes should use misclassification-adjusted methods whenever possible or conduct sensitivity analyses to investigate the possibility of false-positive findings, especially for exposures that may be related to the accuracy of outcome ascertainment.

Bayesian Correction for Exposure Misclassification and Evolution of Evidence in Two Studies of the Association Between Maternal Occupational Exposure to Asthmagens and Risk of Autism Spectrum Disorder.

Inference in epidemiologic studies is plagued by exposure misclassification. Several methods exist to correct for misclassification error. One approach is to use point estimates for the sensitivity (Sn) and specificity (Sp) of the tool used for exposure assessment. Unfortunately, we typically do not know the Sn and Sp with certainty. Bayesian methods for exposure misclassification correction allow us to model this uncertainty via distributions for Sn and Sp. These methods have been applied in epidemiologic literature, but are not considered a mainstream approach, especially in occupational epidemiology. Here, we illustrate an occupational epidemiology application of a Bayesian approach to correct for the differential misclassification error generated by estimating occupational exposures from job codes using a job exposure matrix (JEM). We argue that analyses accounting for exposure misclassification should become more commonplace in the literature.

Differential Misclassification of Disease under Partial-Mouth Sampling.

The effect of misclassification of a cluster-level dichotomous outcome (disease) due to partial-cluster sampling on its association with a dichotomous exposure is investigated. Disease (e.g., chronic periodontitis) is deemed to exist in a cluster (e.g., full mouth) when a condition of interest (e.g., pocket depth or clinical attachment loss exceeding an established threshold) is present in number and pattern across observations (e.g., tooth sites) in the cluster according to a specific criterion. When a subset of observations within each cluster is selected (i.e., partial-mouth sampling), specificity of disease is 100% (in the absence of site-level measurement error), whereas sensitivity is imperfect and generally unknown. Using conditional probability arguments, we investigate disease misclassification under partial-cluster sampling and its impact on the estimated disease-exposure association when the exposure is cluster level and measured without error. When the probability of disease varies by exposure status, outcome misclassification at the cluster level is differential under partial-cluster sampling and depends on 1) the partial recording protocol, including the number of observations sampled and the particular sites selected in a cluster; 2) the joint probability structure of the condition within clusters; and 3) the criterion for disease. A numeric example demonstrates that disease-exposure odds ratios under partial-cluster random sampling can be biased in either direction (toward or away from the null) relative to gold-standard odds ratios under full-cluster sampling. In general, misclassification of disease is differential under partial-cluster sampling. In particular, sensitivity and negative predictive values depend on exposure status, which leads to biased inference. Partial-mouth sampling causes disease misclassification probabilities, including sensitivity, to vary by exposure groups when disease prevalence differs between groups. As a result, disease-exposure associations may be under- or overestimated by standard analysis procedures for periodontal data relative to full-mouth estimates. Procedures that address bias are needed for partial-recording protocols.

Differential misclassification between self-reported status and official HPV vaccination records in Japan: Implications for evaluating vaccine safety and effectiveness

Japan has no national vaccine registry and approximately 1700 municipalities manage the immunization records independently. In June 2013, proactive recommendations for the human papillomavirus (HPV) vaccine were suspended after unconfirmed reports of adverse events following immunization in the media, despite no vaccine safety signal having been raised. Furthermore, studies assessing HPV vaccine safety and effectiveness published post suspension are predominantly based on self-reported information. Our aim was to examine the accuracy of self-reported vaccination status compared with official municipal records. Participants were women aged 20–22 yrs, who were attending for cervical screening in Niigata city. Among the 1230 eligible registrants, vaccine uptake, defined as any dose, was 75.0% and 77.2% according to a self-reported questionnaire and municipal records, respectively. The accuracy rate of self-reported information was as follows: positive predictive value (PPV) was 87.7%; negative predictive value (NPV) was 54.5%; sensitivity was 85.2%; and specificity was 59.8%. The validity of self-reported information was only moderate (Kappa statistic = 0.44, 95% confidence interval 0.37–0.50). This combined with the low NPV may lead to reduced estimation of effectiveness and safety. A more reliable method, such as a national HPV vaccine registry, needs to be established for assessing HPV immunization status in Japan.

Bayesian measurement-error-driven hidden Markov regression model for calibrating the effect of covariates on multistate outcomes: Application to androgenetic alopecia.

Multistate Markov regression models used for quantifying the effect size of state‐specific covariates pertaining to the dynamics of multistate outcomes have gained popularity. However, the measurements of multistate outcome are prone to the errors of classification, particularly when a population‐based survey/research is involved with proxy measurements of outcome due to cost consideration. Such a misclassification may affect the effect size of relevant covariates such as odds ratio used in the field of epidemiology. We proposed a Bayesian measurement‐error‐driven hidden Markov regression model for calibrating these biased estimates with and without a 2‐stage validation design. A simulation algorithm was developed to assess various scenarios of underestimation and overestimation given nondifferential misclassification (independent of covariates) and differential misclassification (dependent on covariates). We applied our proposed method to the community‐based survey of androgenetic alopecia and found that the effect size of the majority of covariate was inflated after calibration regardless of which type of misclassification. Our proposed Bayesian measurement‐error‐driven hidden Markov regression model is practicable and effective in calibrating the effects of covariates on multistate outcome, but the prior distribution on measurement errors accrued from 2‐stage validation design is strongly recommended.

Imbalance p values for baseline covariates in randomized controlled trials: a last resort for the use of p values? A pro and contra debate.

BackgroundResults of randomized controlled trials (RCTs) are usually accompanied by a table that compares covariates between the study groups at baseline. Sometimes, the investigators report p values for imbalanced covariates. The aim of this debate is to illustrate the pro and contra of the use of these p values in RCTs.ProLow p values can be a sign of biased or fraudulent randomization and can be used as a warning sign. They can be considered as a screening tool with low positive-predictive value. Low p values should prompt us to ask for the reasons and for potential consequences, especially in combination with hints of methodological problems.ContraA fair randomization produces the expectation that the distribution of p values follows a flat distribution. It does not produce an expectation related to a single p value. The distribution of p values in RCTs can be influenced by the correlation among covariates, differential misclassification or differential mismeasurement of baseline covariates. Given only a small number of reported p values in the reports of RCTs, judging whether the realized p value distribution is, indeed, a flat distribution becomes difficult. If p values ≤0.005 or ≥0.995 were used as a sign of alarm, the false-positive rate would be 5.0% if randomization was done correctly, and five p values per RCT were reported.ConclusionUse of a low p value as a warning sign that randomization is potentially biased can be considered a vague heuristic. The authors of this debate are obviously more or less enthusiastic with this heuristic and differ in the consequences they propose.