Comparing clinical trial population representativeness to real-world populations: an external validity analysis encompassing 43 895 trials and 5 685 738 individuals across 989 unique drugs and 286 conditions in England

Abstract

Randomised controlled trials (RCTs) inform prescription guidelines, but stringent eligibility criteria exclude individuals with vulnerable characteristics, which we define as comorbidities, concomitant medication use, and vulnerabilities due to age. Poor external validity can result in inadequate treatment decision information. Our first aim was to quantify the extent of exclusion of individuals with vulnerable characteristics from RCTs for all prescription drugs. Our second aim was to quantify the prevalence of individuals with vulnerable characteristics from population electronic health records who are actively prescribed such drugs. In tandem, these two aims will allow us to assess the representativeness between RCT and real-world populations and identify vulnerable populations potentially at risk of inadequate treatment decision information. When a vulnerable population is highly excluded from RCTs but has a high prevalence of individuals actively being prescribed the same medication, there is likely to be a gap in treatment decision information. Our third aim was to investigate the use of real-world evidence in contributing towards quantifying missing treatment risk or benefit through an observational study. We extracted RCTs from ClinicalTrials.gov from its inception to April 28, 2021, and primary care records from the Clinical Practice Research Datalink Gold database from Jan 1, 1998, to Dec 31, 2020. We referred to the British National Formulary to classify prescription drugs into drug categories. We conducted descriptive analyses and quantified RCT exclusion and prevalence of individuals with vulnerable characteristics for comparison to identify populations without treatment decision information. Exclusion and prevalence were assessed separately for different age groups, individual clinical specialities, and for quantities of concomitant conditions by clinical specialities, where multimorbidity was defined as having two or more clinical specialties, and medications prescribed, where polypharmacy was defined as having five or more medications prescribed. Population trends of individuals with multimorbidity or polypharmacy were assessed separately by age group. We conducted an observational cohort study to validate the use of real-world evidence in contributing towards quantifying treatment risk or benefit for patients with dementia on anti-dementia drugs with and without a contraindicated clinical speciality. To do so, we identified the clinical specialities that anti-dementia drug RCTs highly excluded yet had corresponding high prevalence in the real-world population, forming the groups with highest risk of having scarce treatment decision information. Cox regression was used to assess if the risk of mortality outcomes differs between both groups. 43 895 RCTs from ClinicalTrials.gov and 5 685 738 million individuals from primary care records were used. We considered 989 unique drugs and 286 conditions across 13 drug-category cohorts. For the descriptive analyses, the median RCT exclusion proportion across 13 drug categories was 81·5% (IQR 76·7-85·5) for adolescents (aged <18 years), 26·3% (IQR 21·0-29·5) for individuals older than 60 years, 40·5% (IQR 33·7-43·0) for individuals older than 70 years, and 52·9% (IQR 47·1-56·0) for individuals older than 80 years. Multimorbidity had a median exclusion proportion of 91·1% (IQR 88·9-91·8) and median prevalence of 41·0% (IQR 34·9-46·0). Concomitant medication use had a median exclusion proportion of 52·5% (IQR 50·0-53·7) and a median prevalence of 94·3% (IQR 84·3-97·2), and polypharmacy had a median prevalence of 47·7% (IQR 38·0-56·1). Population trends show increasing multimorbidity with age and consistently high polypharmacy across age groups. Populations with cardiovascular or otorhinolaryngological comorbidities had the highest risk of having scarce treatment decision information. For the observational study, populations with cardiovascular or psychiatric comorbidities had highest risk of having scarce treatment decision information. Patients with dementia with an anti-dementia prescription and contraindicated cardiovascular condition had a higher risk of mortality (hazard ratio [HR] 1·20 [95% CI 1·13-1·28 ; p<0·0001]) compared with patients with dementia without a contraindicated cardiovascular condition. Patients with dementia with comorbid delirium (HR 1·25 [95% CI 1·06-1·48]; p<0·0088), intellectual disability (HR 2·72 [95% CI 1·53-4·81]; p=0·0006), and schizophrenia and schizotypal delusional disorders (HR 1·36 [95% CI 1·02-1·82]; p=0·036) had a higher risk of mortality compared with patients with dementia without these conditions. Overly stringent RCT exclusion criteria do not appropriately account for the heterogeneity of vulnerable characteristics observed in real-world populations. Treatment decision information is scarce for such individuals, which might affect health outcomes. We discuss the challenges facing the inclusivity of such individuals and highlight the strength of real-world evidence as an integrative solution in complementing RCTs and increasing the completeness of evidence-based medicine assessments in evaluating the effectiveness of treatment decisions. Wellcome Trust, National Institute for Health Research (NIHR) University College London Hospitals Biomedical Research Centre, NIHR Great Ormond Street Hospital Biomedical Research Centre, Academy of Medical Sciences, and the University College London Overseas Research Scholarship.