Institute For Clinical And Economic Review Reports Research Articles

POSC143 Comparison of Model Structures Used in NICE and ICER Cost Effectiveness Evaluations

The Institute for Clinical and Economic Review (ICER) in the United States and the National Institute for Health and Care Excellence (NICE) in United Kingdom produce evidence-based reports on cost-effectiveness of health technologies to inform policymaking and reimbursement decisions. We aimed to compare the degree to which the structure of cost-effectiveness models submitted to NICE and developed by ICER differ. In 2020–2021, ICER published reports in nine disease areas: multiple myeloma, hereditary angioedema, high cholesterol, atopic dermatitis, sickle cell disease, ulcerative colitis, migraine, non-alcoholic steatohepatitis, and beta thalassemia. We identified 17 NICE technology appraisals in the same disease areas published in 2016–2021. Full-text screening of committee papers and ICER reports was conducted by one investigator and abstracted data was validated by a second. Information on treatment comparators, model structure, and approach were extracted. Evidence review group commentaries were considered. Comparisons were based on the adopted model structure and approach to elicit inputs. Data sources and results were not relevant. Among the disease areas, ICER and NICE used different model structures in three, broadly similar structures in four, and similar structures in two evaluations. Among the models with different structures, NICE were more complex with larger numbers of health states (2/3). Among models with broadly similar structures, NICE implemented approaches that more closely captured the effect of new treatments (2/4) with more complex data elicitation approaches (3/4). Among the models with very similar model structures, the key difference was the means of treatment effect measure. Despite NICE and ICER having similar guidelines for conducting cost-effectiveness assessments, there are differences among model structures within the same disease areas. Availability of trial patient-level data to inform submissions to NICE and manufacturers’ intent to capture treatment effect to the best possible extent could be reasons behind the discordance.

The evolution of ICER's review process for new medical interventions and a critical review of economic evaluations (2018-2019): how stakeholders can collaborate with ICER to improve the quality of evidence in ICER's reports.

BACKGROUND: Since its inception in 2006, the Institute for Clinical and Economic Review (ICER) has rapidly gained influence on drug pricing and reimbursement decisions despite historical resistance to the use of cost-effectiveness thresholds in the US health care system. Although patient groups, physicians, and pharmaceutical manufacturers voiced their concerns about the potential negative effects of increased use of ICER's assessments on patient access to innovative medications, there is little guidance and consensus on how the stakeholders should collaborate with ICER to ensure that its reviews reflect the best clinical and economic evidence. OBJECTIVES: To (1) summarize the evolution of ICER's evaluation procedure, scope, and topics; (2) evaluate the effectiveness of stakeholder engagement approaches; and (3) inform stakeholders of their potential role in collaborating with ICER in estimating the cost-effectiveness of new interventions. METHODS: Publicly available ICER evaluations from 2008 to 2019 were systematically reviewed. Changes in evaluation procedures, scope, and topics were summarized. For evaluations that occurred in 2018 (n = 12) and 2019 (n = 8), key characteristics were extracted from 172 letters documenting interactions between ICER and all stakeholders who provided comments to draft reports. Stakeholder suggestions were analyzed in terms of their effectiveness indicated by ICER's reconsideration of its original cost-effectiveness analysis approach. RESULTS: The number of ICER evaluations increased consistently from 2 to 12 per year between 2008 and 2018 but declined to 8 in 2019. Stakeholder opportunity to engage with ICER increased from 1 to 3 per evaluation between 2008 and 2015. ICER initially focused on reviewing general treatment strategies but shifted its focus to specific pharmaceuticals and medical devices in 2014. In 2018 and 2019, 30% of 172 stakeholder letters resulted in a revision in the base-case analysis (49 comments in 2018, 23 in 2019); nearly half of comments in these letters included specific alternative data or a published article to rationalize recommendations. Other common types of suggestions that resulted in ICER's base-case analysis revisions included comments relating to inconsistent methods used to derive model inputs across different treatments (12/49 in 2018, 5/23 in 2019); clinical justifications (12/49 in 2018, 0/23 in 2019); and justifications based on patient perspectives (1/49 in 2018, 5/23 in 2019). These revisions rarely affected ICER's conclusions on the cost-effectiveness of evaluated interventions. Among the 20 assessments that involved 172 stakeholder engagements in 2018 and 2019, only 2% (n = 3) of the engagements (all from 2018) were associated with a change in the cost-effectiveness conclusion. CONCLUSIONS: Between 2018 and 2019, stakeholders leveraged ICER evaluations as opportunities to promote dialogue for better understanding of the value of technologies. Actionable, evidence-based recommendations were accepted more often than other recommendations. DISCLOSURES: No outside funding supported this study. The authors have no conflicts of interest to disclose. Findings from this study were presented as a poster at Virtual ISPOR, May 17-20, 2021.

PMU49 Patient Heterogeneity and the Limitations of One-Size-Fits-All Value-Based Pricing: A Systematic Review and Future Directions

Because of heterogeneity of treatment effect, decisions based on average measures of cost-effectiveness may lead to incorrect recommendations for specific subsets of the population. This study examined whether the Institute for Clinical and Economic Review (ICER), a leading advocate for wider use of value-based pricing in the U.S., accounts for patient heterogeneity in their assessments. We conducted a systematic review of 8 ICER reports on treatments for a range of conditions (major depressive disorder [MDD], rheumatoid arthritis [RA], ulcerative colitis, high cholesterol, anemia in chronic kidney disease [CKD], and 3 cancers). Within each report we identified the number and type of subgroups examined in CEA along with heterogeneity factors or subgroups mentioned elsewhere in the report but not examined in the CEA. While a variety of potential heterogeneity factors were mentioned in each ICER report, assessment of subgroups within the CEA was typically limited to factors where high-quality trial evidence supported efficacy parameters, either directly or via meta-analysis. Two reports (MDD and RA) did not examine any subgroup in their CEA, 4 reports (all cancers and ulcerative colitis) presented CEA for subgroups defined by 1 factor, and 2 recent reports (high cholesterol and CKD) examined subgroups by 2-3 factors. Examples of heterogeneity factors assessed in CEA included cancer mutation or grade, biologic-naïve vs. experienced, and dialysis-dependent vs. independent. Prices meeting cost-per-QALY thresholds were sometimes reported by these subgroups, but budget-threshold prices were not. Given existing data limitations, ICER recommendations are unlikely to reflect a clinically nuanced, patient-centered value assessment. There is urgent need of a national discussion around the limitations of value assessment when heterogeneity is known to exist but data are incomplete, as well as improving data generation and integration of patient subgroups in cost-effectiveness analysis and value-based pricing, with important implications for equity.

PNS75 The Evolution of Payer Perceptions of Icer

The influence of the Institute for Clinical and Economic Review (ICER) value assessment framework (VAF) on payer decisions has been increasing in recent years. There is limited evidence assessing how ICER’s VAF updates have influenced payer perceptions of ICER reports over time. To assess trends in payer perceptions of ICER’s evaluations over the last 4 years. Double-blinded web-based surveys were fielded in November 2016, November 2018, and October 2020 to individuals from integrated health delivery systems, pharmacy benefits managers, managed care organizations, and specialty pharmacies. There were 55, 54, and 47 payers who completed the surveys in 2016, 2018, and 2020, respectively. Respondents identified clinical outcomes relevant to decision making (62%), validity of the framework (57%), and clear and transparent methodology (53%) as the top strengths of ICER reports in 2020. In prior years, a transparent methodology (53% in 2016 and 56% in 2018) was considered a top strength. Limitations were somewhat consistent with prior years, including not having results at the time of decision making (30% in 2020 vs 28% in 2018 and 45% in 2016). Respondents identified comparative clinical effectiveness (79%) and health-benefit price benchmarks (57%) as the most influential components of ICER reports in 2020, which was consistent with prior years, especially for comparative clinical effectiveness (100% in 2016 and 89% in 2018). The most unclear component of ICER reports has steadily been the other benefits and contextual considerations section (30% in 2020 vs 37% in 2018 and 27% in 2016). While payer perceptions of ICER’s evaluations have evolved, methodology transparency has consistently been reported as a top strength over time, and timeliness of ICER reports continues to be a primary limitation.

PNS48 Do US Commercial Health Plans Cite ICER Reports in Their Specialty Drug Coverage Policies? an Empirical Analysis

The Institute for Clinical and Economic Review (ICER) has emerged as a visible force in US health care. However, despite its rise in prominence, ICER’s influence on health plan decision-making is unclear. Our objective was to determine the frequency that US commercial health plans cite ICER value assessments in their specialty drug coverage policies, and whether citation frequency has increased over time. We used the Tufts Medical Center Specialty Drug Evidence and Coverage (SPEC) Database, which includes specialty drug coverage policies issued by 17 large US commercial health plans. We identified coverage policies for drugs for which ICER had previously published a value assessment. We determined how often plans cited ICER’s value assessments in pertinent coverage policies that were in force in the month of August in four consecutive years (2017-2020). Health plans cited an ICER value assessment in 0.48% (3/622) of 2017 coverage policies for drugs for which ICER had previously published a value assessment, 0.36% (3/833) of 2018 policies, 1.32% (15/1139) of 2019 policies, and 1.70% (26/1529) of 2020 policies. The number of plans that cited an ICER value assessment in at least one of their coverage policies increased from three in 2017 to 11 in 2020. In 2020, eight ICER value assessments were cited by multiple plans, three assessments by a single plan, and 25 assessments by no plans. ICER’s assessment of Duchenne muscular dystrophy therapies was the value assessment cited most frequently by health plans (cited by five plans). CONCLUSION: Health plans infrequently cite ICER value assessments in their coverage policies, although citation frequency has increased over time. The included health plans cited ICER value assessments with different frequencies. Some ICER value assessments were cited by more health plans than others.

Are Drugs Priced in Accordance With Value? A Comparison of Value-Based and Net Prices Using Institute for Clinical and Economic Review Reports

ObjectivesThe Institute for Clinical and Economic Review (ICER) is an independent organization that reviews drugs and devices with a focus on emerging agents. As part of their evaluation, ICER estimates value-based prices (VBP) at $50 000 to $150 000 per quality-adjusted life-year (QALY) gained thresholds. We compared actual estimated net prices to ICER-estimated VBPs. MethodsWe reviewed ICER final evidence reports from November 2007 to October 2020. List prices were combined with average discounts obtained from SSR Health to estimate net prices. If a drug had been evaluated more than once for the same indication, only the more recent VBP was included. ResultsA total of 34 ICER reports provided unique VBPs for 102 drugs. The net price of 81% of drugs exceeded the $100 000 per QALY VBP and 71% exceeded the $150 000 per QALY VBP. The median change in net price needed to reach the $150 000 per QALY VBP was a 36% reduction. The median decrease in net price needed was highest for drugs targeting rare inherited disorders (n = 15; 62%) and lowest for cardiometabolic disorders (n = 6; 162% price increase). The reduction in net prices needed to reach ICER-estimated VBPs was higher for drugs evaluated for the first approved indication, rare diseases, less competitive markets, and if the drug approval occurred before the ICER report became available. ConclusionNet prices are often above VBPs estimated by ICER. Although gaining awareness among decision makers, the long-term impact of ICER evaluations on pricing and access to new drugs continues to evolve.

The use of real-world evidence in ICER's scoping process and clinical evidence assessments.

BACKGROUND: There has been growing interest in using real-world evidence (RWE) for health technology assessment (HTA) in the United States. The Institute for Clinical and Economic Review (ICER) is an independent U.S.-based HTA organization that focuses primarily on pharmaceuticals. RWE is used to inform ICER's scoping and comparative clinical effectiveness (CCE) assessments, but the extent to which it is used has not been quantified. OBJECTIVE: To systematically evaluate use of RWE in the scoping and CCE assessment sections of the ICER HTA reports on pharmaceuticals. METHODS: We reviewed all ICER reports of pharmaceuticals published between January 2014 and June 2019. We examined the average number of instances and the proportion of RWE use in the scoping documents to inform the population, intervention, comparator, outcome, setting, or timing (PICOTS) elements of the appraisal. We also examined the average number of instances and the proportion of RWE use in the CCE assessments to inform effectiveness, safety, or treatment patterns. Finally, we evaluated use of RWE in clinical guidelines that were cited in the CCE assessments. RESULTS: In ICER scoping documents, the mean (SD) number of instances of RWE use was 3.8 (3.7) per document (55% for outcomes, 20% for population, 14% for comparator, 11% for intervention, and 0% for timing and setting). In ICER CCE assessments, the mean (SD) number of instances per assessment was 0.7 (0.5) per drug (53% for effectiveness, 44% for safety, and 3% for treatment patterns). In clinical guidelines used in ICER reports, the mean (SD) number of instances of RWE use was 1.6 (2.3) per drug per guideline (41% for effectiveness, 30% for safety, and 29% for treatment patterns). CONCLUSIONS: RWE was frequently used in the ICER scoping process, particularly to inform selection of outcomes. RWE was used infrequently in ICER CCE assessments, while more often used to inform effectiveness, safety, and treatment patterns in relevant clinical guidelines. There are opportunities to increase the use of RWE in U.S. HTA processes. DISCLOSURES: This study was supported by the Health Tech Fund, University of Washington School of Pharmacy, which was created through unrestricted support from several health care industry companies. Veenstra and Carlson report grant support from the Institute for Clinical and Economic Review outside the submitted work. Carlson reports personal fees from Bayer, Allergan, and Galderma outside the submitted work. Jiao, Lee, and Devine report no support outside the submitted work.

HTA and economics in the United States: a systematic review of ICER reports to evaluate trends, identify factors associated with recommendations, and understand implications.

BACKGROUND: The Institute for Clinical and Economic Review (ICER) is a prominent health technology assessment (HTA) entity in the United States that considers costs and applies economic analyses to derive price-based recommendations. ICER continues to adjust its value framework, yet discussion persists regarding whether ICER methodologies align with established research standards. This work evaluates ICER assessments relative to those standards, providing a benchmark with the release of ICER's most recent value framework update. OBJECTIVES: To evaluate ICER economic assessments for trends, factors related to recommendations, and quality for use in U.S. decision making. METHODS: We evaluated all ICER final evidence reports published between 2006 and August 31, 2019, with regard to base-case result trends over time, pricing sources, comparator selection, analytic perspectives, model uncertainty, how modeling results aligned with ICER's determinations of value for money, and comparison of ICER methodological approaches with established modeling standards. Analyses were stratified by time period, where appropriate, to account for changes in ICER's framework over time. RESULTS: Of 58 ICER final evidence reports, 47 used the most commonly reported outcome (cost per quality-adjusted life-year [QALY]); ICER-developed models evaluated 131 interventions and comparators with 238 base-case results. Pricing sources for ICER reports became more standardized in 2017, although sources were not associated with the likelihood of falling below ICER's cost-effectiveness thresholds. In 30% of base-case analyses (n = 72), ICER did not use a clinical comparator, although reasonable treatments were available. In modified societal perspectives scenarios applied in later assessments, 75% of analyses (n = 76) included productivity but did not specify how it was quantified. Reports did not explain how sensitivity and scenario analyses were selected or implications of results. ICER value for money determinations generally aligned with cost-effectiveness results, although 2 of 33 (6%) interventions ranked as low value and 3 of 5 (60%) interventions ranked as low-moderate value, met a $150,000 per QALY threshold, and 14 of 37 (38%) moderate-value interventions exceeded this threshold; the most common rationale was related to national budget impact. CONCLUSIONS: While some progress has been made, further improvement is needed to ensure that ICER assessments address the most relevant questions for target audiences, adhere to established research standards, and are reported in a manner that can be readily interpreted and applied to policymaking. DISCLOSURES: No outside funding supported this study. The authors have nothing to disclose.

PNS136 ICER AND THE USE OF MEDICAID DATA

ICER (Institute for Clinical and Economic review) is an independent, non-profit research institute that seeks to establish the clinical effectiveness and value of drugs. ICER has committed to use Real World Evidence (RWE), such as payer reimbursement data, to quantify the burden of illness on the US overall health system. Commercial claims studies are now commonly cited sources, however there is a lack of RWE based on reimbursement data for Medicaid and other government programs. Medicaid is a federal-state assistance program that serves low-income individuals of all ages, as well as about half (47%) of the 13.3 million children with special health care needs. In particular, many children with high cost rare diseases would be eligible for Medicaid and this cost of care would comprise a large proportion of the overall health system spending. Thus, Medicaid RWE data is necessary to accurately evaluate the potential cost offsets of new drugs on the health system as a whole, especially in the cases of rare diseases. We reviewed all completed ICER assessments (N=40, as of January 15, 2020) and identified the data sources for the cost-related model inputs. We flagged reviews utilizing Medicaid RWE to quantify the existing burden on the US health system. Of the 40 completed ICER reports, 4 (10%) utilized any Medicaid data. This is likely attributable to a lack of published data analyses that incorporate Medicaid data. 9 of the 40 reviewed ICER reports were for rare diseases (<200,000 cases per year in the US), but none of these utilized Medicaid data. The majority of ICER assessments do not include the healthcare costs currently paid for by Medicaid, so may underestimate the value of new drugs to the US overall health system, particularly in the case of high-cost rare diseases.

PNS121 DOES THE CHOICE OF INDEPENDENT EXPERT COMMITTEE AFFECT ICER OUTCOMES?

Objectives: The Institute for Clinical and Economic Review (ICER) publishes reports on the clinical and economic value of drugs for US payers. Each report is subject to a public meeting by one of three independent expert committees (the California Technology Assessment Forum [CTAF], Midwest Comparative Effectiveness Public Advisory Council [CEPAC], and New England CEPAC) discusses the report and votes on the evidence presented. This research compares and contrasts outcomes by committee. Methods: All publicly available final ICER reports of pharmaceuticals (01/01/2014-12/31/2019) were screened and key data extracted. Results: 38 ICER reports of pharmaceuticals were identified (CTAF: 11; Midwest CEPAC: 12; New England CEPAC: 15), encompassing 110 drugs in 78 conditions/subgroups. Clinical ratings were provided for 277 drug comparisons (CTAF: 41; Midwest CEPAC: 46; New England CEPAC: 190). For the CTAF, Midwest CEPAC, New England CEPAC, respectively, 19 (46%), 23 (50%), and 33 (17%) drug comparisons achieved a rating of A, B, or B+; 3 (7%), 9 (20%), and 80 (42%) achieved a rating of C, C+, C-, or D; 19 (46%), 14 (30%), and 77 (41%) achieved a rating of P/I or I. Cost-utility assessments were undertaken for 117 treatment settings (CTAF: 35; Midwest CEPAC: 32; New England CEPAC: 50). At a willingness-to-pay threshold of $150,000/QALY, 11 therapies (31%) were deemed cost-effective/cost-saving by the CTAF, 9 (28%) by the Midwest CEPAC, and 21 (42%) by the New England CEPAC. Of the 51 votes for healthcare value, most drugs had a majority vote of low or intermediate value: CTAF: 9 (90%); Midwest CEPAC: 12 (86%); New England CEPAC: 22 (81%). Conclusions: There are generally few differences between ICER committee outcomes, although there are some variances, which are likely due to therapy area focus. Considering healthcare value, ICER committees are generally consistent in reporting that most drugs provide low to intermediate value.

Nonsense on Stilts - Part 1: The ICER 2020-2023 Value Assessment Framework for Constructing Imaginary Worlds.

Previous commentaries in the Formulary Evaluation section of INNOVATIONS in Pharmacy have pointed to the lack of credibility in modeled claims for cost-effectiveness and associated recommendations for pricing and access by the Institute for Clinical and Economic Review (ICER). The principal objection to ICER reports has been that their modeled claims fail the standards of normal science: they are best seen as pseudoscience. The purpose of this latest commentary is to provide a critique of the recently released ICER 2020 Value Assessment Framework (VAF). Although ICER has taken upon itself the pole position in health technology assessments and recommendations for product pricing in the US health care system, the incremental, lifetime cost-per-QALY modeling methodology should not be taken seriously. The creation of imaginary modeled worlds, built entirely from assumption, fails the demarcation test between science and pseudoscience. The ICER evidence reports are best seen as the health technology assessment equivalent of 'intelligent design' in counterpoint to 'natural selection'. It is surprising, therefore, that health care decision makers should take ICER's recommendations seriously as providing 'approximate information' for formulary decision making. What is not appreciated is that the claims made by ICER lack credibility, are impossible to evaluate and lack the ability to be replicated across treatment settings. Indeed, the models presented under the guise of a 'state of the art' value assessment were never intended to support evaluable claims. We have no idea and will never know if they are right or if they are wrong. ICER's position becomes even more untenable once the models presented are assessed in detail. Without in any way supporting the ICER methodology, it is worth noting that all too often ICER's claims for incremental QALYs in specific models are based upon what appears to be, from the limited evidence presented, a casual and ad hoc assemblage of utility scores from diverse constructs. This is a critical weakness given the role attributed by ICER to the modeled cost-per-QALY claims as central to ICERs imaginary value assessment. ICER also overlooks the fact that the utility scores it captures from the literature to populate its imaginary reference case world lack objectivity. They are ordinal rather than interval measures. To apply these manifest scores to time spent in a disease stage and then aggregate these over different disease stages is nonsensical. The critical issue is one of instrument development. The case made here is for the application of Rasch Measurement Theory (RMT) to construct a unidimensional instrument with interval properties, in this case from the needs fulfillment construct of quality of life (QoL). Unless an instrument meets RMT standards in its development, the logic of Rasch modeling to achieve fundamental measurement standards means that other scales are, by definition, ordinal. It is absurd to 'assume' they are interval. RMT is designed to create instruments to evaluate change and test hypotheses. In the absence of instruments that have RMT properties, the cost-per-QALY reference case modelling meme collapses. It is an analytical dead end. If we are to support a meaningful scientific program to discover new facts to support health care delivery and improve the lives of patients, caregivers and their families, then ICER should be put to one side.

More Unnecessary Imaginary Worlds - Part 1: The Institute for Clinical and Economic Review's Evidence Report on Janus Kinase (JAK) Inhibitors in Rheumatoid Arthritis.

Previous commentaries in the Formulary Evaluation section of INNOVATIONS in Pharmacy have pointed to the lack of credibility in modeled claims for cost-effectiveness and associated recommendations for pricing by the Institute for Clinical and Economic Review (ICER). The principal objection to ICER reports has been that their modeled claims fail the standards of normal science: they are best seen as pseudoscience. The purpose of this latest commentary is to consider the recently released ICER evidence report for Janus Kinase (JAK) Inhibitors. As ICER continues, in the case of JAK Inhibitors, to apply its modeled cost utility framework with consequent recommendations for pricing adjustments, these recommendations also lack credibility. In contrast with previous ICER evidence reports, the present report adopts only a 12-month timeframe, one due, in large part, to ICER being unable to justify assumptions to drive its construction of imaginary worlds beyond 12 months. This commentary emphasizesagain, why the ICER methodology fails to meet the standards of normal science. Claims made by ICER for the competing JAK Inhibitor therapies lack credibility, are impossible to evaluate, let alone replicate across treatment settings. Even so, it is important to examine a number of key elements in the ICER invention of the 12-month JAK Inhibitor imaginary world. While this does not imply any degree of acceptance of the ICER methodology, one element that merits particular attention is thefailure of the ICER modeling to meet logically defensible measurement standards in its application of generic health related quality of life (HRQoL) ordinal metrics to create its QALY claims. The failure to meet the required standards of fundamental measurement means that the cost-per-QALY claims are invalid. This raises the issue of the application of Rasch Measurement Theory (RMT) in instrument development and the potential role of patient centric outcome (PCO) instruments that represent the patient voice in value claims. The case made here is that the ICER approach should be abandoned as an unnecessary distraction. If we are to meet standards for the discovery of new facts in therapy response then our focus must be on proposing credible, evaluable and replicable claims within disease states. Instruments, such as the Rheumatoid Arthritis Quality of Life (RAQoL)questionnaire that build on the common construct that QoL is the extent to which human needs are fulfilled should be the basis for value claims. HRQoL Instruments that are clinically focused and reflect the value calculus of providers and not patients in measuring response by symptoms and activity limitations are irrelevant. This puts to one side the belief that incremental cost-per-QALY models, the construction of imaginary worlds are, in any sense, a 'gold standard'; a meme embraced by the health technology assessment profession. Claims for incremental cost per QALY outcomes and recommendations for pricing and access driven by willingness to pay thresholds are irrelevant to formulary decisions.

Yet another Ersatz World: The ICER Final Evidence Report for Additive Cardiovascular Therapies.

Previous commentaries in the Formulary Evaluation section of INNOVATIONS in Pharmacy have pointed to the lack of credibility in modeled claims for cost-effectiveness and associated recommendations for pricing by the Institute for Clinical and Economic Review (ICER). The principal objection to ICER reports has been that their modeled claims fail the standards of normal science: they are best seen as pseudoscience. The purpose of this latest commentary is to consider the recently released ICER report for Additive Cardiovascular Disease therapies. This report should not be taken seriously in its claims for cost-effectiveness and pricing in cardiovascular disease (CVD). The analytical framework applied by ICER fails to meet the standards of normal science in demarcating science from pseudoscience. Irrespective of the value judgements and recommendations of an ICER report, these lack credibility. They were never intended to be evaluable and replicable across treatment settings. The claims made are constructed, driven by assumption, and should be put to one side by health system decision makers. In this review the focus is on to the ICER modeled estimates of utility scores in CVD, the insistence on utilizing a generic utility algorithm (the EQ-5D-3L) and the consequent quality adjusted life year (QALY) estimates. Two issues are raised that will be the subject of future commentaries: the lack of appreciation of fundamental measurement and (ii) the importance of the patient voice in benefit claims. Given the importance in the ICER methodology of QALYS, the ad hoc nature of the ordinal utilities introduced to the cardiovascular model must raise concerns over the role the ICER evidence report may play in health care decision-making. These concerns extend to the claim by ICER that, on ICER's own affordability threshold for individual new molecular entities, the anticipated uptake of these therapies may raise questions of overall affordability. Again, we are dealing with an arbitraryconstruct that may adversely impact patient access.

ICER, ISPOR AND QALYs: Tales of Imaginary Worlds.

Previous commentaries in the Formulary Evaluation section of INNOVATIONS in Pharmacy have pointed to the lack of credibility in modeled claims for cost-effectiveness and associated recommendations for pricing by the Institute for Clinical and Economic Review (ICER). The principal objection to ICER reports has been that their modeled claims fail the standards of normal science: they are best seen as pseudoscience. The purpose of this latest commentary is twofold: first, to review the latest report by the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) on standards and value claims for modeling imaginary cost-per-quality life year(QALY) worlds in health technology assessments and (ii) given ICER's commitment to ISPOR standards, to consider thecredibility of ICER's QALYs. The concern is that ICER gives the impression that there is a common and agreed utility metric to support QALY constructs and that there is an agreed standard for creating QALYs within the imaginary reference case meme. The purpose of this commentary is to emphasize that there is no 'gold standard' QALY, let alone a 'gold standard' utility metric. A QALY is what you assume the QALY should be; it is an artificial construct which allows any number of competing QALYs to be generated for the same target population under different imaginary constructs. Similar objections apply to the application of willingness-to-pay thresholds to support ICER's business case as the sole arbiter, in pole position, of health technology assessments to support pricing recommendations. In the US. A threshold has to be defined for the QALYs and costs captured, by assumption, within the model. Even for the same target patient population a $50,000 cost per QALY threshold will yield different pricing recommendations depending upon the assumptions driving the QALY estimate, such as choice of utility metric, time spent by stage of disease, frequency and severity of adverse events and costs. While this is an obvious point, it appears not to occur to those who, with a limited technical understanding of how the ICER model is constructed (and ICER is not given to extended technical explanations) take at face value the ICER imaginary construct and the QALY value judgements. Of course, it is also beside the point that the ICER reference case imaginary world was never intended to support empirical assessments of the claims made; we have no idea of whether it is right, if it is wrong and, over the lifetime perspective, we will never know and were never intended to know.

Does the Institute for Clinical and Economic Review Revise Its Findings in Response to Industry Comments?

BackgroundThe Institute for Clinical and Economic Review (ICER) has gained prominance through its work conducting health technology assessments of pharmaceuticals in the United States. ObjectiveTo understand the influence of industry comments on pharmaceutical value assessments conducted by ICER. MethodsWe reviewed 15 ICER reports issued from 2017 through 2019. We quantified ICER’s revisions to its cost-effectiveness analysis (CEA) estimates between release of its draft and revised evidence reports and whether ratios shifted across ICER-specified categories of high, intermediate, or low value. We also reviewed industry-submitted comments recommending revision to ICER’s CEAs, noting ICER’s response as no change, text revised, assumption(s) revised, or conclusion revised. We evaluated each comment in terms of clarity, whether it offered an alternative to ICER’s approach, and whether it characterized the expected impact of revision on ICER’s analysis. ResultsWe identified 53 ICER-reported ratios. Of these, 45 (84.9%) changed between the draft and revised report, but 26 changes (57.8%) were small (<10%). Six ratios shifted across value categories. We identified 256 industry comments recommending that ICER revise its CEA. Of these, 159 (62%) lacked clarity, 145 (57%) offered no alternative, and 243 (95%) did not characterize their impact on ICER’s estimated ratio. Ninety-one comments (35.5%) caused ICER to revise its assumptions, but only 5 (2.0%) caused ICER to revise its conclusions. Four of these 5 comments characterized their impact on ICER’s findings. ConclusionsChanges in ICER’s estimates of cost-effectiveness between its draft and revised evidence reports are generally modest. Greater precision in industry comments could increase the influence of industry critiques, thus enhancing the dialogue around pharmaceutical value.

Another Rush to Judgment: The Imaginary Worlds of ICER and Recommendations in Duchenne Muscular Dystrophy.

Previous commentaries in the Formulary Evaluation section of INNOVATIONS in Pharmacy have pointed to the lack of credibility in modeled claims for cost-effectiveness and associated recommendations for pricing by the Institute for Clinical and Economic Review (ICER). The principal objection to ICER reports has been that their modeled claims fail the standards of normal science: they are best seen as pseudoscience. The purpose of this latest commentary is to consider the recently released ICER report for Duchennemuscular dystrophy (DMD). As ICER has continued in the case of DMD to apply its modeled cost utility framework with consequent recommendations for pricing adjustments, these recommendations also lack credibility. This commentary emphasizes again not only why the ICER methodology fails to meet the standards of normal science but to point to the importance in rare diseases for accelerated approval, while recognizing that evidence generation will continue.While this assessment of the ICER DMD model does not imply any support for this methodology, a key point is the application of quality of life measures which fail to capture the experience of patients with DMD and, importantly, the interests of both patients and caregivers. While ICER would argue that even with a limited evidence base it is important to address issues of pricing and access for new products, their reports are used as justification for coverage and reimbursement by insurers and health system decision makers without recognition of their lack of scientific merit. This rush to judgement by ICER must raise concerns about potentially adverse formulary decisions that result in access restrictions on new products. If ICER is to make a contribution to the entry of new products in the health market place then it should consider an alternative methodology that generates claims that are empirically evaluable in a timeframe relevant to health decision makers. As it stands, ICER's recommendation should be rejected. This is not a research program that meets accepted scientific standards but one that relies on the willingness of an audience to accept the proposition that evidence is constructed not discovered.

Challenges with Forecasting Budget Impact: A Case Study of Six ICER Reports

BackgroundPayers frequently rely on budget impact model (BIM) results to help determine drug coverage policy and its effect on their bottom line. It is unclear whether BIMs typically overestimate or underestimate real-world budget impact. ObjectiveWe examined how different modeling assumptions influenced the results of 6 BIMs from the Institute for Clinical and Economic Review (ICER). Study DesignRetrospective analysis of pharmaceutical sales data. MethodsFrom ICER reports issued before 2016, we collected estimates of 3 BIM outputs: aggregate therapy cost (ie, cost to treat the patient population with a particular therapy), therapy uptake, and price. We compared these against real-world estimates that we generated using drug sales data. We considered 2 classes of BIM estimates: those forecasting future uptake of new agents, which assumed “unmanaged uptake,” and those describing the contemporaneous market state (ie, estimates of current, managed uptake and budget impact for compounds already on the market). ResultsDifferences between ICER's estimates and our own were largest for forecasted studies. Here, ICER's uptake estimates exceeded real-world estimates by factors ranging from 7.4 (sacubitril/valsartan) to 54 (hepatitis C treatments). The “unmanaged uptake” assumption (removed from ICER's approach in 2017) yields large deviations between BIM estimates and real-world consumption. Nevertheless, in some cases, ICER's BIMs that relied on current market estimates also deviated substantially from real-world sales data. ConclusionsThis study highlights challenges with forecasting budget impact. In particular, assumptions about uptake and data source selection can greatly influence the accuracy of results.

PHP24 - Icer Reports – 4 Years of Cost/Qaly Drug Evaluations in the USA… what Does the Future Hold?

In 2014, the Institute for Clinical and Economic Review (ICER) became the focus of attention for publishing a report concluding that new drug treatments for Hepatitis C offered low value for US healthcare payers based on a clinical and economic assessment. This research systematically analyzes ICER reports of drugs and discusses their potential future role in drug pricing. All publically available final ICER reports of pharmaceuticals (01/01/2014-12/31/2017) were screened and key data extracted. 19 ICER reports of pharmaceuticals were identified, encompassing 65 drugs in 37 conditions/subgroups. 56 panel votes of their clinical benefits deemed 64% clinically effective by majority vote. 39% were clearly (≥80% votes) clinically effective, with 25% clearly not clinically effective. Clinical ratings were provided for 58 drugs : 52% achieved ≥B rating, 17% C–D and 31% I or P/I. A cost-utility assessment was undertaken for 49 drug:indication pairings. Even at the highest WTP threshold of 150,000/QALY, only 29% therapies were deemed cost-effective, with a mean 46% discount required for an ICER<$150,000. Of 30 votes on the “care value” of a product, based on their clinical and cost-effectiveness, 33% were rated as low value, 3% low/intermediate, 43% intermediate, 7% intermediate/high, and 13% high. Of 11 votes on the “health system value” of products, based upon their anticipated budget impact and affordability, 100% were deemed low value. ICER reports have generally concluded that assessed therapies offer important clinical benefits but generally do not justify their economic impact. These reports currently do not influence coverage/pricing decisions but there is anecdotal evidence of insurers using these in negotiations. For example they were formally cited as part of Medicare Part B payment model reform discussions, indicating ICER reports may play a greater role in future reforms to US drug reimbursement as payers struggle to manage increasing costs.