Indirect treatment comparisons and biologics

Abstract

There has been much recent excitement among both patients and clinicians over the development and approval of new biologic medications, mAbs, for asthma, the first new class of medications targeted at pathobiological pathways since antileukotrienes became available in the 1990s.1Drazen J.M. Israel E. O'Byrne P.M. Treatment of asthma with drugs modifying the leukotriene pathway.New Engl J Med. 1999; 340: 197-206Crossref PubMed Scopus (768) Google Scholar, 2Chu E.K. Drazen J.M. Asthma: one hundred years of treatment and onward.Am J Respir Crit Care Med. 2005; 171: 1202-1208Crossref PubMed Scopus (67) Google Scholar Clinicians want to know how to use these medications and which of those with similar mechanisms are likely to be most effective for their patients.3Manka L.A. Wechsler M.E. Selecting the right biologic for your patients with severe asthma.Ann Allergy Asthma Immunol. 2018; 121: 406-413Abstract Full Text Full Text PDF PubMed Scopus (39) Google Scholar These medications are very expensive (Table I), and their duration of use, long-term benefits, and adverse effects are unknown. In addition, biologics with similar mechanisms, such as blocking the actions of IL-5, have not been directly compared for safety or efficacy in head-to-head clinical trials. However, 2 recent publications have constructed indirect comparisons based on separate placebo-controlled clinical trials.4Casale T.B. Pacou M. Mesana L. Farge G. Sun S.X. Castro M. Reslizumab compared with benralizumab in patients with eosinophilic asthma: a systematic literature review and network meta-analysis.J Allergy Clin Immunol Pract. 2019; 7: 122-130Abstract Full Text Full Text PDF PubMed Scopus (43) Google Scholar, 5Busse W. Chupp G. Nagase H. Albers F.C. Doyle S. Shen Q. et al.Anti-IL-5 treatments in patients with severe asthma by blood eosinophil thresholds: indirect treatment comparison.J Allergy Clin Immunol. 2019; 143: 190-200Abstract Full Text Full Text PDF PubMed Scopus (150) Google Scholar Interestingly, the results are not completely congruent.Table IAverage wholesale price∗The average wholesale price (AWP) is the average value at which wholesalers sell drugs to physicians, pharmacies, and other customers. The AWP is a generally accepted standard measure for calculating the cost of a medication. It is typically 20% greater than the value for which a manufacturer sells their products to distributors and large customers. of one dose and for a 52-week year of mepolizumab, reslizumab, and benralizumab for a 70-kg adultBiologicDoseCost of 1 dose for a 70-kg subjectCost for a 52-wk yearMepolizumab100 mg SQ q4W$3,442.40$44,751.20Reslizumab3 mg/kg IV q 4$107.52/mL (∼$2150)$27,950.00Benralizumab30 mg SQ q4W for 3 doses and then q8W$5,702.5348,471.51, year 1;$42,768.98, thereafterIV, Intravenous; Q, every; SQ, subcutaneous; W, week.∗ The average wholesale price (AWP) is the average value at which wholesalers sell drugs to physicians, pharmacies, and other customers. The AWP is a generally accepted standard measure for calculating the cost of a medication. It is typically 20% greater than the value for which a manufacturer sells their products to distributors and large customers. Open table in a new tab IV, Intravenous; Q, every; SQ, subcutaneous; W, week. An indirect comparison estimates the difference between 2 treatments, say X and Y, through a common comparator. Registration trials for all new drugs include placebo, which makes it the logical choice for a common comparator. Thus the difference between the treatments can be represented as follows:TX−TY=[TX−PX]−[TY−PY]−[TX−PY],where T denotes treatment and P denotes placebo. Under this framework, [TX − PX] and [TY − PY] can be estimated from existing placebo-controlled trials for the 2 treatments. For obvious reasons, however, trials comparing PX and PY will never be done, which necessitates an assumption that the difference between PX and PY (whatever it is) is small relative to [TX − PX] and [TY − PY]. However, even if it is reasonable to assume that the difference between PX and PY is very small, there is a more fundamental condition that must also be met for the indirect comparison to be valid. The justification for the indirect comparison rests on the assumption that the results of a hypothetical head-to-head trial comparing treatments X and Y can be approximated by using results from separate trials of X and Y with a common comparator. This would be reasonable if these separate trials were done under the same conditions and in the same patient populations. A network meta-analysis (NMA) builds evidence by combining results from head-to-head trials (direct evidence) with results from common comparator trials (indirect evidence), whereas a traditional meta-analysis includes only head-to-head trials.6Rouse B. Chaimani A. Li T. Network meta-analysis: an introduction for clinicians.Intern Emerg Med. 2017; 12: 103-111Crossref PubMed Scopus (258) Google Scholar Although some degree of variation across study populations is acceptable in a traditional meta-analysis, it can be disastrous in an NMA. Because an NMA combines 2 different types of trials, it necessitates the assumption of “consistency,” which means that the indirect and direct evidence is congruent; that is, if the indirect evidence from the common comparator trials included in the NMA indicates that treatment X is superior to treatment Y, then such differences also should be seen in head-to-head trials. Unfortunately, these recent NMAs do not include any head-to-head trials because none have been done. Therefore although NMA is a statistically sound methodology that has been applied in many settings, there is significant vulnerability in these recent publications because it is not possible to objectively assess the consistency assumption, and we are left to evaluate it subjectively. Establishing that the trials were done under equivalent conditions would go a long way in this regard. PICOS (Population, Intervention, Comparator, Outcomes, Study Design) is an ideal framework for this assessment (Table II). With respect to the patient population aspect, it is not absolutely necessary for the trials included in the NMA to be identical in all characteristics, but it is critical that they be very similar in all characteristics that are important for trials of biologics in asthmatic patients because those are the ones most likely to affect consistency. How does one determine these characteristics? An obvious place to start is inclusion criteria and suspected predictors of asthma outcomes and treatment responses. Both of the recent NMAs address this issue by comparing the baseline characteristics of the patient populations included in their analyses. Busse et al5Busse W. Chupp G. Nagase H. Albers F.C. Doyle S. Shen Q. et al.Anti-IL-5 treatments in patients with severe asthma by blood eosinophil thresholds: indirect treatment comparison.J Allergy Clin Immunol. 2019; 143: 190-200Abstract Full Text Full Text PDF PubMed Scopus (150) Google Scholar take the step of formally evaluating which of these characteristics are likely to be effect modifiers for biologics. For example, if studies differ by the inclusion of patients with different blood eosinophil counts (≥0 vs ≥350 cells/μL), their response to the drug could be different. However, the bar they set is so high (ie, “conclusive evidence of effect modification”) that they could be overlooking important suspected modifiers that have not been proved yet. In any case, if the patient populations in the trials included in an NMA differ in terms of important characteristics, then we have reason to be concerned about the consistency assumption. One approach, which is applied in both reports, is to reduce this concern by extracting subgroups of patients that are similar across all of the trials included. Although this does help increase our confidence in the consistency assumption, it narrows the generalizability of the comparison to a smaller population.Table IIThe PICOS framework, outlined in this table, is used to assess whether the component studies in meta-analyses are sufficiently similar. This table includes examples of elements used in the component studies that must be comparedElementPPopulationAgeSexCriteria used to define severe eosinophilic asthmaBaseline eosinophil countsBaseline asthma controlBaseline asthma-related quality of lifeBaseline FEV1 percent predictedBaseline medications, including dose of inhaled corticosteroidBaseline asthma severity (GINA level)Number of exacerbations in year before enrollmentSmoking historyIInterventionAnti–IL-5 biologicDuration of treatmentCComparatorPlaceboOther asthma medicationsComorbiditiesOOutcomesClinically significant exacerbationsAsthma controlAsthma-related quality of lifeEmergency department visits/hospitalizationsSStudy typeRandomized controlled trialGINA, Global Initiative for Asthma. Open table in a new tab GINA, Global Initiative for Asthma. With respect to the intervention and comparator aspects of PICOS, there is also reason for concern about these NMAs. For example, both NMAs include trials that differ by duration of intervention therapy, ranging from 15 to 56 weeks.4Casale T.B. Pacou M. Mesana L. Farge G. Sun S.X. Castro M. Reslizumab compared with benralizumab in patients with eosinophilic asthma: a systematic literature review and network meta-analysis.J Allergy Clin Immunol Pract. 2019; 7: 122-130Abstract Full Text Full Text PDF PubMed Scopus (43) Google Scholar, 5Busse W. Chupp G. Nagase H. Albers F.C. Doyle S. Shen Q. et al.Anti-IL-5 treatments in patients with severe asthma by blood eosinophil thresholds: indirect treatment comparison.J Allergy Clin Immunol. 2019; 143: 190-200Abstract Full Text Full Text PDF PubMed Scopus (150) Google Scholar In addition, although all of the trials included in these NMAs used placebos, that does not necessarily make them equivalent comparators. Because asthma outcomes are affected by background controller medications (eg, inhaled corticosteroids/long-acting β-agonists), we cannot be confident that the placebo arms of the component trials offer comparable results. In addition to patient characteristics, interventions, and comparators, compatibility of outcomes must also be examined.7Busse W.W. Morgan W.J. Taggart V. Togias A. Asthma outcomes workshop: overview.J Allergy Clin Immunol. 2012; 129: S1-S8Abstract Full Text Full Text PDF PubMed Scopus (103) Google Scholar For these NMAs, they concern lung function, asthma control, and exacerbations. Lung function measurement is highly standardized and thus an ideal outcome. With respect to asthma control, the 2 most commonly used tools are the Asthma Control Questionnaire, which queries symptoms over the past week,8Juniper E.F. O'Byrne P.M. Guyatt G.H. Ferrie P.J. King D.R. Development and validation of a questionnaire to measure asthma control.Eur Respir J. 1999; 14: 902-907Crossref PubMed Scopus (1815) Google Scholar and the Asthma Control Test,9Nathan R.A. Sorkness C.A. Kosinski M. Schatz M. Li J.T. Marcus P. et al.Development of the asthma control test: a survey for assessing asthma control.J Allergy Clin Immunol. 2004; 113: 59-65Abstract Full Text Full Text PDF PubMed Scopus (2022) Google Scholar which asks for symptoms over the past 4 weeks. In these NMA studies the Asthma Control Questionnaire was uniformly used. Exacerbations, on the other hand, were defined less uniformly across the trials included in these NMAs. Exacerbations are an inherently difficult outcome to standardize because they entail subject assessments at points. First, the patient must make the decision to seek medical care, and second, a physician who might or might not be familiar with the study protocol must make the decision to initiate systemic corticosteroids. Most clinical trials try to address this by supplying patient asthma action plans and specifying criteria for initiating systemic corticosteroids, but significant variation remains. With respect to the study design component of PICOS, the fact that patients in both the placebo and active arms improve suggests another mechanism of benefit: that these drugs, which must be given under medical supervision, are a form of directly observed therapy. This is because patients are seen regularly by medical personnel to receive the study medication, any illness is likely to be recognized more quickly, and treatment is likely to be initiated more promptly. With all of the differences in the trials available for inclusion, it is not surprising that these 2 NMAs comparing mAbs report different results.4Casale T.B. Pacou M. Mesana L. Farge G. Sun S.X. Castro M. Reslizumab compared with benralizumab in patients with eosinophilic asthma: a systematic literature review and network meta-analysis.J Allergy Clin Immunol Pract. 2019; 7: 122-130Abstract Full Text Full Text PDF PubMed Scopus (43) Google Scholar, 5Busse W. Chupp G. Nagase H. Albers F.C. Doyle S. Shen Q. et al.Anti-IL-5 treatments in patients with severe asthma by blood eosinophil thresholds: indirect treatment comparison.J Allergy Clin Immunol. 2019; 143: 190-200Abstract Full Text Full Text PDF PubMed Scopus (150) Google Scholar As we try to learn about and compare these new biologics, it is critical to understand the limitations of the NMA. Most important, they must include some head-to-head trials making direct comparisons to rigorously and objectively evaluate the critical assumption of consistency. In addition, different design choices can lead to inclusion of different patients and result in different outcomes that might be valid for the cohort studied but not provide information on patients with different characteristics. Those design choices (PICOS) should be detailed in research reports. It is also important not to lose sight of the fact that there are other aspects of caring for patients with asthma that NMAs cannot consider. For example, they do not provide information on patients' and clinicians’ goals, priorities, and beliefs. Initial studies on new medications tend to focus on patients most likely to benefit from the medication and might not include patients with comorbidities, such as hypertension, diabetes, or obesity or exposures to tobacco, pollutants, and other toxic agents. These considerations tell us that clinicians and patients must wait until more patients are exposed to these new medications for longer periods and under real-life conditions to fully appreciate their benefits and adverse events. Finally, problems of cost must be overcome. Reslizumab Compared with Benralizumab in Patients with Eosinophilic Asthma: A Systematic Literature Review and Network Meta-AnalysisThe Journal of Allergy and Clinical Immunology: In PracticeVol. 7Issue 1PreviewThe interaction of IL-5 with its receptor on eosinophils increases the activation and maintenance of eosinophils; blocking this interaction reduces asthma symptoms in patients with the eosinophilic phenotype. Reslizumab, which binds to IL-5, and benralizumab, which targets the IL-5 receptor α subunit, have not been compared in head-to-head trials. Full-Text PDF Anti–IL-5 treatments in patients with severe asthma by blood eosinophil thresholds: Indirect treatment comparisonJournal of Allergy and Clinical ImmunologyVol. 143Issue 1PreviewThree anti–IL-5 pathway–directed therapies are approved for use in patients with severe eosinophilic asthma (SEA); however, no head-to-head comparison data are available. Full-Text PDF Open AccessCorrigendumJournal of Allergy and Clinical ImmunologyVol. 143Issue 3PreviewWith regard to the Editorial published in the January 2019 issues of JACI and JACI: In Practice entitled “Indirect treatment comparisons and biologics” (J Allergy Clin Immunol 2019;143:84-6), the authors have notified both Journals that the last terms in the equation in the first column contained an error. The equation should read: TX−Ty = [TX−PX] − [TY−PY] + [PX−PY] rather than TX−Ty = [TX−PX] − [TY−PY] − [TX−PY] as printed. Additionally, a dollar sign was missing from Table I, last column, 3rd line, which should be modified to read: $48,471.51, year 1. Full-Text PDF