Evidence Review Group Research Articles

Assessment of quality of data submitted for NICE technology appraisals over two decades

BackgroundThe National Institute for Health and Care Excellence (NICE) pioneered the Health Technology Assessment (HTA) processes and methodologies. Technology appraisals (TAs) focus on pharmaceutical products and clinical and economic data,...

PP113 Common Methodological Issues In Systematic Reviews Supporting Single Technology Appraisal Submissions To The National Institute for Health and Care Excellence

IntroductionThis project aimed to identify methodological issues reported by Evidence Review Groups (ERGs) in the systematic reviews (SR) supporting single technology appraisal (STA) submissions to the National Institute for Health and Care Excellence (NICE). STA submissions contain SRs related to the clinical and cost-effectiveness of the proposed intervention and NICE require the methods for these reviews to be clearly detailed in the submission. The intention of this project was to identify methodological aspects of submissions where companies may need additional guidance or support to provide the evidence required for efficient and effective decision-making and in turn facilitate timely access to clinical innovation.MethodsFrom 2019, 61 STAs were identified from the NICE website, of which 46 were included. We extracted information about the data requests or clarification questions raised by the ERG in relation to the methodological section of both the clinical and cost effectiveness SRs reported in the STA. We then categorized these data and grouped by theme to determine the most common methodological issues faced by companies. We did not assess whether comments made by the ERG were accurate or justified.ResultsFor both clinical and cost-effectiveness SRs, the most frequent clarification questions arose from the search methods section, specifically seeking information about missing intervention or comparator terms, the use of search filters and search platforms. Clarification questions were also commonly asked about the appropriateness of interventions and comparators. There were very few clarification questions asked about screening, data extraction or risk of bias assessment.ConclusionsCompanies looking to submit an STA should align their submission methodology to established best practice guidance for systematic reviewing to ensure their methods are fit for purpose and avoid unnecessary delays to the STA process. Consistency with the PRISMA reporting standards would help ensure that the ERG is provided with the information needed to assess the appropriateness of the STA methodology and is likely to reduce the need for clarification questions.

SA66 The Changing Landscape of Indirect-Treatment Comparison Models for NSCLC in NICE Appraisals: Do Complex Models Fare Well?

Violation of the proportional-hazards (PH) assumption may lead to an underestimation of treatment effects when conventional therapies are compared to immunotherapies. When the PH assumption was violated, several health technology assessments in the UK for drugs in non-small cell lung cancer (NSCLC) used complex indirect treatment comparison (ITC) models to demonstrate treatment benefits. The purpose of this review is to describe how such ITC methodologies have evolved over time in tandem with drug innovation, as well as to assess how well complex models were accepted by Evidence Review Groups (ERG).

Pegcetacoplan for Treating Paroxysmal Nocturnal Haemoglobinuria: An Evidence Review Group Perspective of a NICE Single Technology Appraisal.

As part of the Single Technology Appraisal (STA) process, the UK National Institute for Health and Care Excellence (NICE) invited Apellis Pharmaceuticals/Sobi to submit evidence for the clinical and cost effectiveness of pegcetacoplan versus eculizumab and pegcetacoplan versus ravulizumab for treating paroxysmal nocturnal haemoglobinuria (PNH) in adults whose anaemia is uncontrolled after treatment with a C5 inhibitor. The Liverpool Reviews and Implementation Group at the University of Liverpool was commissioned as the Evidence Review Group (ERG). The company pursued a low incremental cost-effectiveness ratio (ICER) Fast Track Appraisal (FTA). This was a form of STA processed in a shorter time frame and designed for technologies with company base-case ICER < £10,000 per quality-adjusted life-year (QALY) gained and most plausible ICER < £20,000 per QALY gained. This article summarises the ERG's review of the company's evidence submission, and the NICE Appraisal Committee's (AC's) final decision. The company presented clinical evidence from the PEGASUS trial that assessed the efficacy of pegcetacoplan versus eculizumab. At Week 16, patients in the pegcetacoplan arm had statistically significantly greater change from baseline in haemoglobin levels and a higher rate of transfusion avoidance than patients in the eculizumab arm. Using the PEGASUS trial and Study 302 data (a non-inferiority trial that assessed ravulizumab versus eculizumab), the company conducted an anchored matching-adjusted indirect comparison (MAIC) to indirectly estimate the efficacy of pegcetacoplan versus ravulizumab. The company identified key differences between trial designs and populations that could not be adjusted for using anchored MAIC methods. The company and ERG agreed that the anchored MAIC results were not robust and should not inform decision making. In the absence of robust indirect estimates, the company assumed that ravulizumab had equivalent efficacy to eculizumab in the PEGASUS trial population. Results from the company base-case cost-effectiveness analysis showed that treatment with pegcetacoplan dominated eculizumab and ravulizumab. The ERG considered that the long-term effectiveness of pegcetacoplan was uncertain and ran a scenario assuming that after 1 year the efficacy of pegcetacoplan would be the same as eculizumab; treatment with pegcetacoplan continued to dominate eculizumab and ravulizumab. The AC noted that treatment with pegcetacoplan had lower total costs than treatment with eculizumab or ravulizumab because it is self-administered and reduces the need for blood transfusions. If the assumption that ravulizumab has equivalent efficacy to eculizumab does not hold, then this will affect the estimate of the cost effectiveness of pegcetacoplan versus ravulizumab; however, the AC was satisfied that the assumption was reasonable. The AC recommended pegcetacoplan as an option for the treatment of PNH in adults who have uncontrolled anaemia despite treatment with a stable dose of a C5 inhibitor for ≥ 3 months. Pegcetacoplan was the first technology recommended by NICE via the low ICER FTA process.

Oral Azacitidine for Maintenance Treatment of Acute Myeloid Leukaemia After Induction Therapy: An Evidence Review Group Perspective of a NICE Single Technology Appraisal.

The National Institute for Health and Care Excellence (NICE) invited the manufacturer (Celgene) of oral azacitidine (ONUREG), as part of the Single Technology Appraisal (STA) process, to submit evidence for the clinical effectiveness and cost-effectiveness of oral azacitidine for maintenance treatment of acute myeloid leukaemia (AML) after induction therapy compared with watch-and-wait plus best supportive care (BSC) and midostaurin. Kleijnen Systematic Reviews Ltd, in collaboration with Maastricht University Medical Centre+, was commissioned to act as the independent Evidence Review Group (ERG). This paper summarises the company submission (CS), presents the ERG's critical review on the clinical and cost-effectiveness evidence in the CS, highlights the key methodological considerations and describes the development of the NICE guidance by the Appraisal Committee. In the QUAZAR AML-001 trial, oral azacitidine significantly improved overall survival (OS) versus placebo: median OS gain of 9.9months (24.7months versus 14.8months; hazard ratio (HR) 0.69 (95% CI 0.55-0.86), p < 0.001). The median time to relapse was also better for oral azacitidine, and the incidences of TEAEs were similar for the two arms. The company excluded two of the comparators listed in the scope, low-dose cytarabine and subcutaneous azacitidine, informed only by clinical expert opinion, leaving only best supportive care (BSC) and midostaurin for the FLT3-ITD and/or FLT3-TKD (FLT3 mutation)-positive subgroup. An ITC comparing oral azacitidine to midostaurin as maintenance therapy in the appropriate subgroup demonstrated that the OS and relapse-free survival (RFS) HRs were favourable for oral azacitidine when compared with midostaurin. However, in the only available trial of midostaurin as maintenance treatment in AML that was used for this ITC, subjects were not randomised at the maintenance phase, but at induction, which posed a substantial risk of bias. The revised and final probabilistic incremental cost-effectiveness ratio (ICER) presented by the company, including a commercial arrangement, was £32,480 per quality-adjusted life year (QALY) gained for oral azacitidine versus watch-and-wait plus BSC. Oral azacitidine was dominant versus midostaurin in the FLT-3 subgroup. The ERG's concerns included the approach of modelling haematopoietic stem cell transplantation (HSCT), the generalisability of the population and the number of cycles of consolidation therapy pre-treatment in the QUAZAR AML-001 trial to UK clinical practice, and uncertainty in the relapse utility. The revised and final ERG base case resulted in a similar probabilistic ICER of £33,830 per QALY gained versus watch-and-wait plus BSC, but with remaining uncertainty. Oral azacitidine remained dominant versus midostaurin in the FLT-3 subgroup. After the second NICE appraisal committee meeting, the NICE Appraisal Committee recommended oral azacitidine (according to the commercial arrangement), within its marketing authorisation, as an option for maintenance treatment for AML in adults who are in complete remission, or complete remission with incomplete blood count recovery, after induction therapy with or without consolidation treatment, and cannot have or do not want HSCT.

Trastuzumab Deruxtecan for Treating HER2-Positive Unresectable or Metastatic Breast Cancer After Two or More Anti-HER2 Therapies: An Evidence Review Group Perspective of a NICE Single Technology Appraisal.

The National Institute for Health and Care Excellence (NICE) provides guidance to improve health and social care in England and Wales. NICE invited Daiichi Sankyo to submit evidence for the use of trastuzumab deruxtecan (T-DXd) for treating human epidermal growth factor 2 (HER2)-positive unresectable or metastatic breast cancer (UBC/MBC) after two or more anti-HER2 therapies, in accordance with NICE's Single Technology Appraisal process. The Liverpool Reviews and Implementation Group, part of the University of Liverpool, was commissioned to act as the Evidence Review Group (ERG). This article summarises the ERG's review of the evidence submitted by the company and provides an overview of the NICE Appraisal Committee's (AC's) final decision made in May 2021. Results from the company's base-case fully incremental analysis showed that, compared with T-DXd, eribulin and vinorelbine were dominated and the incremental cost-effectiveness ratio (ICER) per quality-adjusted life year (QALY) gained versus capecitabine was £47,230. The ERG scenario analyses generated a range of ICERs, with the highest being a scenario relating to a comparison of T-DXd versus capecitabine (£78,142 per QALY gained). The ERG considered that due to a lack of appropriate clinical effectiveness evidence, the relative effectiveness of T-DXd versus any comparator treatment could not be determined with any degree of certainty. The NICE AC agreed that the modelling of overall survival was highly uncertain and concluded that treatment with T-DXd could not be recommended for routine use within the National Health Service (NHS). T-DXd was, however, recommended for use within the Cancer Drugs Fund, provided Managed Access Agreement conditions were followed.

Abemaciclib in Combination with Endocrine Therapy for Adjuvant Treatment of Hormone Receptor-Positive, HER2-Negative, Node-Positive Early Breast Cancer: An Evidence Review Group Perspective of a NICE Single Technology Appraisal.

The National Institute for Health and Care Excellence (NICE) invited the manufacturer (Eli Lilly) of abemaciclib (Verzenios) to submit evidence for the clinical and cost effectiveness of this drug in combination with endocrine therapy (ET) for the treatment of adult patients with hormone receptor (HR)-positive, human epidermal growth factor receptor 2 (HER2)-negative, node-positive early breast cancer at high risk of recurrence, as part of the Institute's Single Technology Appraisal (STA) process. Kleijnen Systematic Reviews Ltd, in combination with Newcastle University, was commissioned to act as the independent Evidence Review Group (ERG). This paper summarised the Company Submission (CS), presents the ERG's critical review of the clinical and cost-effectiveness evidence in the CS, highlights the key methodological considerations, and describes the development of the NICE guidance by the Appraisal Committee. The ERG produced a critical review of the evidence for the clinical and cost-effectiveness evidence in the CS and also independently searched for relevant evidence and modified the manufacturer decision analytic model to examine the impact of altering some of the key assumptions. A systematic literature review identified the MonarchE trial, an ongoing, open-label, randomised, double blind trial involving 5637 people comparing abemaciclib in combination with ET versus ET alone. The trial included two cohorts that used different inclusion criteria to define high risk of recurrence. The ERG considered Cohort 1 as an adequate representation of this population and the AC concluded that Cohort 1 was generalisable to National Health Service clinical practice. Trial results showed improvements in invasive disease-free survival for the abemaciclib arm, which was considered an appropriate surrogate outcome. The ERG believed that the modelling structure presented in the de novo economic model by the company was appropriate but highlighted several areas of uncertainty that had the potential to have a significant impact on the resulting incremental cost-effectiveness ratio (ICER). Areas of uncertainty included the extrapolation of long-term survival curves, the duration of treatment effect and treatment waning, and the proportion of patients who receive other CDK4/6 treatments for metastatic disease after receiving abemaciclib. ICER estimates were £9164 per quality-adjusted life-year gained for the company's base-case and £17,810 for the ERG's base-case. NICE recommended abemaciclib with ET as an option for the adjuvant treatment of HR-positive, HER2-negative, node-positive early breast cancer at high risk of recurrence.

Pralsetinib for RET Fusion-Positive Advanced Non-small-Cell Lung Cancer: An Evidence Review Group Perspective of a NICE Single Technology Appraisal.

The National Institute for Health and Care Excellence (NICE) invited the manufacturer (Roche) of pralsetinib (Gavreto®), as part of the single technology appraisal (STA) process, to submit evidence for the clinical effectiveness and cost effectiveness of pralsetinib for the treatment of adult patients with rearranged during transfection (RET) fusion-positive advanced non-small-cell lung cancer (NSCLC) not previously treated with a RET inhibitor. Kleijnen Systematic Reviews Ltd, in collaboration with University Medical Center Groningen, was commissioned to act as the independent Evidence Review Group (ERG). This paper summarizes the company submission (CS), presents the ERG's critical review of the clinical and cost-effectiveness evidence in the CS, highlights the key methodological considerations, and describes the development of the NICE guidance by the Appraisal Committee. The CS reported data from the ARROW trial. ARROW is a single-arm, multicenter, non-randomized, open-label, multi-cohort study in patients with RET fusion-positive NSCLC and other advanced solid tumors. The CS included both untreated and pre-treated RET fusion-positive NSCLC patients, among other disease types. The comparators in the untreated population were pembrolizumab + pemetrexed + chemotherapy and pembrolizumab monotherapy. The comparators for the pre-treated population were docetaxel monotherapy, docetaxel + nintedanib, and platinum-based chemotherapy ± pemetrexed. As no comparators were included in ARROW, an indirect treatment comparison was conducted to estimate relative effectiveness. The ERG's concerns included the immaturity of data, small sample size, and lack of comparative safety evidence. The ERG considers the clinical evidence presented to be insufficiently robust to inform the economic model. Even when all the ERG preferred assumptions were implemented in the model, uncertainty remained on a number of issues, such as the appropriateness of the hazard ratios and the methods and data used to derive them, long-term efficacy of pralsetinib, and direct evidence for health-related quality of life (HRQoL). NICE did not recommend pralsetinib within its marketing authorization for treating RET fusion-positive advanced non-small-cell lung cancer (NSCLC) in adults who have not had a RET inhibitor before. The uncertainty of the clinical evidence and the estimates of cost effectiveness were too high to be considered a cost-effective use of NHS resources. Therefore, pralsetinib was not recommended for routine use.

Filgotinib for Treating Moderately to Severely Active Ulcerative Colitis: An Evidence Review Group Perspective of a NICE Single Technology Appraisal.

The National Institute for Health and Care Excellence invited the manufacturer (Galapagos) of filgotinib (Jyseleca®), as part of the Single Technology Appraisal process, to submit evidence for the clinical effectiveness and cost effectiveness of filgotinib for treating moderately to severely active ulcerative colitis in adults who have had an inadequate response, loss of response or were intolerant to a previous biologic agent or conventional therapy. Kleijnen Systematic Reviews Ltd, in collaboration with Maastricht University Medical Centre+, was commissioned to act as the independent Evidence Review Group. This paper summarises the company submission, presents the Evidence Review Group's critical review on the clinical and cost-effectiveness evidence in the company submission, highlights the key methodological considerations and describes the development of the National Institute for Health and Care Excellence guidance by the Appraisal Committee. The company submission included one relevant study for the comparison of filgotinib versus placebo: the SELECTION trial. As there was no head-to-head evidence with any of the comparators, the company performed two separate network meta-analyses, one for the biologic-naïve population and one for the biologic-experienced population, and for both the induction and maintenance phases. The Evidence Review Group questioned the validity of the maintenance network meta-analysis because it assumed all active treatments to be comparators in this phase, which is not in line with clinical practice. The economic analysis used a number of assumptions that introduced substantial uncertainty, which could not be fully explored, for instance, the assumption that a risk of loss of response would be independent of health state and constant over time. Company and Evidence Review Group results indicate that at its current price, and disregarding confidential discounts for comparators and subsequent treatments, filgotinib dominates some comparators (golimumab and adalimumab in the company base case, all but intravenous and subcutaneous vedolizumab in the Evidence Review Group's base case) in the biologic-naïve population. In the biologic-experienced population, filgotinib dominates all comparators in both the company and the Evidence Review Group's base case. Results should be interpreted with caution as some important uncertainties were not included in the modelling. These uncertainties were mostly centred around the maintenance network meta-analysis, loss of response, health-related quality-of-life estimates and modelling of dose escalation. The National Institute for Health and Care Excellence recommended filgotinib within its marketing authorisation, as an option for treating moderately to severely active ulcerative colitis in adults when conventional or biological treatment cannot be tolerated, or if the disease has not responded well enough or has stopped responding to these treatments, and if the company provides filgotinib according to the commercial arrangement.

HTA137 Evidence Review Group Criticism of Clinical Effectiveness Systematic Reviews in NICE Oncology Submissions

Clinical effectiveness evidence is a key component of companies’ health technology submissions to the UK National Institute for Health and Care Excellence (NICE) and has to be sought via systematic literature reviews (SLRs). To identify where, and better understand why, NICE Evidence Review Groups (ERGs) have criticised SLRs on clinical effectiveness (clinical SLRs), we reviewed their responses to recent oncology-related submissions.

Fenfluramine for Treating Dravet Syndrome: An Evidence Review Group Perspective of a NICE Single Technology Appraisal.

Fenfluramine, tradename Fintepla®, was appraised within the National Institute for Health and Care Excellence (NICE) single technology appraisal (STA) process as Technology Appraisal 808. Within the STA process, the company (Zogenix International) provided NICE with a written submission and a mathematical health economic model, summarising the company's estimates of the clinical effectiveness and cost-effectiveness of fenfluramine for patients with Dravet syndrome (DS). This company submission (CS) was reviewed by an evidence review group (ERG) independent of NICE. The ERG, Kleijnen Systematic Reviews in collaboration with Maastricht University Medical Centre, produced an ERG report. This paper presents a summary of the ERG report and the development of the NICE guidance. The CS included a systematic review of the evidence for fenfluramine. From this review the company identified and presented evidence from two randomised trials (Study 1 and Study 1504), an open-label extension study (Study 1503) and 'real world evidence' from a prospective and retrospective study. Both randomised trials were conducted in patients up to 18 years of age with DS, whose seizures were incompletely controlled with previous anti-epileptic drugs. A Bayesian network meta-analysis was performed to compare fenfluramine with cannabidiol plus clobazam. There was no evidence of a difference between any doses of fenfluramine and cannabidiol in the mean convulsive seizure frequency (CSF) rate during treatment. However, fenfluramine increased the number of patients achieving ≥50% reduction in CSF frequency from baseline compared to cannabidiol. The company used an individual-patient state-transition model (R version 3.5.2) to model cost-effectiveness of fenfluramine. The CSF and convulsive seizure-free days were estimated using patient-level data from the placebo arm of the fenfluramine registration studies. Subsequently, a treatment effect of either fenfluramine or cannabidiol was applied. Utility values for the economic model were obtained by mapping Pediatric Quality of Life Inventory data from the registration studies to EuroQol-5D-3L Youth (EQ-5D-Y-3L). The company included caregiver utilities in their base-case, as the severe needs of patients with DS have a major impact on parents and caregivers. There were several key issues. First, the company included caregiver utilities in the model in a way that when patients in the economic model died, the corresponding caregiver utility was also set to zero. Second, the model was built in R statistical software, resulting in transparency issues. Third, the company assumed the same percentage reduction for convulsive seizure days as was estimated for CSF. Fourth, during the final appraisal committee meeting, influential changes were made to the model that were not in line with the ERG's preferences (but were accepted by the appraisal committee). The company's revised and final incremental cost effectiveness ratio(ICER) in line with committee preferences resulted in fenfluramine dominating cannabidiol. Fenfluramine was recommended as an add-on to other antiepileptic medicines for treating seizures associated with DS in people aged 2 years and older in the National Health Service(NHS).

Belimumab for Treating Active Autoantibody-Positive Systemic Lupus Erythematosus: An Evidence Review Group Perspective of a NICE Single Technology Appraisal.

As part of its Single Technology Appraisal (STA) process, the National Institute for Health and Care Excellence (NICE) invited the manufacturer (GlaxoSmithKline [GSK]) of Benlysta (belimumab) to submit evidence regarding its clinical and cost effectiveness, for the review and possible extension of a previously conditionally approved intravenous formulation of belimumab for the treatment of active autoantibody-positive systemic lupus erythematosus (SLE). Kleijnen Systematic Reviews Ltd, in collaboration with Maastricht University Medical Centre+, was commissioned to act as the independent Evidence Review Group (ERG). This paper summarises the company submission (CS), presents the ERG’s critical review of the clinical and cost-effectiveness evidence in the CS, highlights the key methodological considerations, and describes the development of the NICE guidance by the NICE Appraisal Committee.This appraisal is different to the previous appraisal in three ways: (1). This appraisal expands its definition of ‘high disease activity’. (2). In TA397, belimumab was approved, with a managed access arrangement (MAA), for adults only. This appraisal includes subjects aged 5 years or older. (3). The original appraisal included an intravenous formulation only, but the current appraisal also includes a new subcutaneous formulation in the form of a prefilled pen.The company was required to collect real-world data from the British Isles Lupus Assessment Group Biologics Register (BILAG-BR), including data on the efficacy, safety, and effect on health-related quality of life of belimumab versus rituximab. This appraisal considers these data as well as additional clinical trial evidence presented in the company’s updated submission to address uncertainties identified during the original appraisal. The ERG identified three major concerns with the evidence presented on the clinical effectiveness in the current submission; namely, short follow-up in the main comparative trials (BLISS-SC, BLISS-52 and BLISS-76); using the propensity score-matching (PSM) analysis in calibrating the cost-effectiveness model can severely bias the results in favour of belimumab; and BILAG-BR data are not suitable for a comparison of belimumab with rituximab.The main issue in the economic analysis was the uncertainty about long-term disease activity progression and resulting organ damage. The company’s approach of calibrating modelled organ damage to longer-term data analysed using the PSM analysis was methodologically inappropriate. The final analysis comparing belimumab with standard treatment for the intravenous formulation resulted in an incremental cost-effectiveness ratio of £12,335 per quality-adjusted life-year (QALY) gained and £30,278 per QALY gained in the company’s and ERG’s base-case analyses, respectively. For the subcutaneous formulation, the final analysis resulted in £8480 per QALY gained and £29,313 per QALY gained in the company’s and ERG’s base-case analyses, respectively. NICE recommended belimumab in both intravenous and subcutaneous formulations as an add-on treatment option for active autoantibody-positive SLE in the HDA-2 subgroup.

HTA72 Survival Extrapolation for Immuno-Oncology Drugs: Pessimism or Optimism? a Targeted Literature Review and a Validation-Based Case Study

The objectives of this study are (1) examine the justification of selecting complex survival extrapolation models among ones suggested in the DSU Technical Support Document 21 and how Evidence Review Groups and appraisal committees responded to the use of complex models, (2) assess the accuracy of the used and unused methods in the NICE technical appraisal 581 to establish whether alternative methods could have provided better prediction.

HTA46 Key Issues in Health Economic Analysis in Nice Highly Specialised Technology Appraisals

To determine key themes and issues identified by Evidence Review Groups (ERG) and the NICE committee during the NICE Highly Specialised Technology (HST) Appraisal process. Also, to explore the relationship between issues that limit the ability of the NICE committee to approve a product, as well as looking at ways that companies mitigate uncertainty in their appraisals.

A Review of Economic Models Submitted to NICE's Technology Appraisal Programme, for Treatments of T1DM & T2DM.

Background: In the UK, 4.7 million people are currently living with diabetes. This is projected to increase to 5 million by 2025. The direct and indirect costs of T1DM and T2DM are rising, and direct costs already account for approximately 10% of the National Health Service (NHS) budget. Objective: The aim of this review is to assess the economic models used in the context of NICE’s Technology Appraisals (TA) Programme of T1DM and T2DM treatments, as well as to examine their compliance with the American Diabetes Association’s (ADA) guidelines on computer modelling. Methods: A review of the economic models used in NICE’s TA programme of T1DM and T2DM treatments was undertaken. Relevant TAs were identified through searching the NICE website for published appraisals completed up to April 2021. The review also examined the associated Evidence Review Group (ERG) reports and Final Appraisal Documents (FAD), which are publicly accessible. ERG reports were scrutinised to identify major issues pertaining to the economic modelling. The FAD documents were then examined to assess how these issues reflected on NICE recommendations. Results: Overall, 10 TAs pertaining to treatments of T1DM and T2DM were identified. Two TAs were excluded as they did not use economic models. Seven of the 8 included TAs related to a novel class of oral antidiabetic drugs (OADs), gliflozins, and one to continuous subcutaneous insulin infusion (CSII) devices. There is a lack of recent, robust data informing risk equations to enable the derivation of transition probabilities. Despite uncertainty surrounding its clinical relevance, bodyweight/BMI is a key driver in many T2DM-models. HbA1c’s reliability as a predictor of hard outcomes is uncertain, chiefly for macrovascular complications. The external validity of T1DM is even less clear. There is an inevitable trade-off between the sophistication of models’ design, their transparency and practicality. Conclusion: Economic models are essential tools to support decision-making in relation to market access and ascertain diabetes technologies’ cost effectiveness. However, key structural and methodological issues exist. Models’ shortcomings should be acknowledged and contextualised within the framework of technology appraisals. Diabetes medications and other technologies should also be subject to regular and consistent re-appraisal to inform disinvestment decisions. Artificial intelligence could potentially enhance models’ transparency and practicality.

Incremental benefits of novel pharmaceuticals in the UK: a cross-sectional analysis of NICE technology appraisals from 2010 to 2020

ObjectivesTo evaluate the incremental value of new drugs across disease areas receiving favourable coverage decisions by the UK’s National Institute for Health and Care Excellence (NICE) over the past decade.Design,...

POSC143 Comparison of Model Structures Used in NICE and ICER Cost Effectiveness Evaluations

The Institute for Clinical and Economic Review (ICER) in the United States and the National Institute for Health and Care Excellence (NICE) in United Kingdom produce evidence-based reports on cost-effectiveness of health technologies to inform policymaking and reimbursement decisions. We aimed to compare the degree to which the structure of cost-effectiveness models submitted to NICE and developed by ICER differ. In 2020–2021, ICER published reports in nine disease areas: multiple myeloma, hereditary angioedema, high cholesterol, atopic dermatitis, sickle cell disease, ulcerative colitis, migraine, non-alcoholic steatohepatitis, and beta thalassemia. We identified 17 NICE technology appraisals in the same disease areas published in 2016–2021. Full-text screening of committee papers and ICER reports was conducted by one investigator and abstracted data was validated by a second. Information on treatment comparators, model structure, and approach were extracted. Evidence review group commentaries were considered. Comparisons were based on the adopted model structure and approach to elicit inputs. Data sources and results were not relevant. Among the disease areas, ICER and NICE used different model structures in three, broadly similar structures in four, and similar structures in two evaluations. Among the models with different structures, NICE were more complex with larger numbers of health states (2/3). Among models with broadly similar structures, NICE implemented approaches that more closely captured the effect of new treatments (2/4) with more complex data elicitation approaches (3/4). Among the models with very similar model structures, the key difference was the means of treatment effect measure. Despite NICE and ICER having similar guidelines for conducting cost-effectiveness assessments, there are differences among model structures within the same disease areas. Availability of trial patient-level data to inform submissions to NICE and manufacturers’ intent to capture treatment effect to the best possible extent could be reasons behind the discordance.

PP299 A Framework And Analysis Assessing The Impact Of Patient-Centered Outcome Evidence In HTA Appraisals

IntroductionThe importance of patient-centered outcome (PCO) evidence is increasingly recognized, but its inclusion in Health Technology Assessment (HTA) submissions remains inconsistent. We explored the impact of PCO evidence on HTA decision-making.MethodsA framework was developed to assess the impact of PCO evidence (excluding EQ-5D) on HTA appraisals. An impact rating was determined by reviewing company, committee and Evidence Review Group (ERG) opinion. This was applied to publicly available appraisal documents (National Institute for Health and Care Excellence [NICE]: 8; Scottish Medicines Consortium [SMC]: 2) in a pilot study. The framework was then refined and applied to a larger dataset.ResultsPCO evidence had ‘substantial impact’ in 3/8 NICE and 1/2 SMC appraisals, and ‘some impact’ in those remaining. PCO evidence informed the cost-effectiveness model in 2/8 NICE and 1/2 SMC submissions, and was considered superior to EQ-5D evidence in one NICE and one SMC submission. The ERG considered PCO evidence relevant to decision-making in 5/8 NICE appraisals. PCO evidence was mentioned in guidance for 7/10 appraisals (deemed relevant in 5/10). In one assessment, committee comments were notably more favorable than ERG comments. Larger dataset analysis results provided further insights to the pilot study.ConclusionsThe framework allows a systematic approach to evaluating the impact of PCO evidence on HTA appraisals.BL, AP, DGB and NY are employees of Symmetron Ltd, which received funding from Pfizer UK in connection with the development of this manuscript. KH, HT, SLC and JB are employees of Pfizer UK. This study was sponsored by Pfizer UK.

UNITI Mobile-EMI-Apps for a Large-Scale European Study on Tinnitus.

More and more observational studies exploit the achievements of mobile technology to ease the overall implementation procedure. Many strategies like digital phenotyping, ecological momentary assessments or mobile crowdsensing are used in this context. Recently, an increasing number of intervention studies makes use of mobile technology as well. For the chronic disorder tinnitus, only few long-running intervention studies exist, which use mobile technology in a larger setting. Tinnitus is characterized by its heterogeneous patient's symptom profiles, which complicates the development of general treatments. In the UNITI project, researchers from different European countries try to unify existing treatments and interventions to cope with this heterogeneity. One study arm (UNITI Mobile) exploits mobile technology to investigate newly implemented interventions types, especially within the pan-European setting. The goals are to learn more about the validity and usefulness of mobile technology in this context. Furthermore, differences among the countries shall be investigated. Practically, two native intervention apps have been developed for UNITI and the mobile study arm, which pose features not presented so far in other apps of the authors. Along the implementation procedure, it is discussed whether these features might leverage similar types of studies in future. Since instruments like the mHealth evidence reporting and assessment checklist (mERA), developed by the WHO mHealth technical evidence review group, indicate that aspects shown for UNITI Mobile are important in the context of health interventions using mobile phones, our findings may be of a more general interest and are therefore being discussed in the work at hand.

Mogamulizumab for Previously Treated Mycosis Fungoides and Sézary Syndrome: An Evidence Review Group Perspective of a NICE Single Technology Appraisal.

The National Institute for Health and Care Excellence (NICE) invited the manufacturer (Kyowa Kirin) of mogamulizumab (Poteligeo®), as part of the single technology appraisal process, to submit evidence for its clinical and cost-effectiveness for previously treated mycosis fungoides (MF) and Sézary syndrome (SS). Kleijnen Systematic Reviews Ltd, in collaboration with Maastricht University Medical Centre, was commissioned to act as the independent evidence review group (ERG). This paper summarises the company submission (CS), presents the ERG’s critical review of the clinical and cost-effectiveness evidence in the CS, highlights the key methodological considerations and describes the development of the NICE guidance by the Appraisal Committee. Based on a systematic literature review, one randomised controlled trial, MAVORIC, was identified showing favourable results in patients with MF and SS. However, MAVORIC compared mogamulizumab to vorinostat, which is not standard care in the NHS, and there is uncertainty due to the study design, specifically crossover of patients. Based on a “naïve comparison of results from the vorinostat arm of the MAVORIC study and the physician’s choice arm (methotrexate or bexarotene i.e. United Kingdom [UK] standard treatments) of the ALCANZA study as well as comparison to Phase II bexarotene data”, the company considered vorinostat to be “a reasonable proxy for current standard of care in the NHS”. The ERG considered, based on the limited data available, that the comparability of vorinostat (MAVORIC) and physician’s choice (ALCANZA) could not be established. In response to the Appraisal Consultation Document, the company provided an unanchored matched adjusted indirect comparison (MAIC) of mogamulizumab with UK standard care by analysing Hospital Episode Statistics (HES) data. However, given the high risk of bias of an unanchored MAIC, these results needed to be regarded with a considerable degree of caution. The economic analysis suffered from uncertainty because there was no trial evidence on the comparator in the England and Wales National Health Service (NHS), and it was unclear to what extent the trial (MAVORIC) comparator (vorinostat) was comparable to standard care, referred to as established clinical management (ECM) in the NHS. The evidence for overall survival had not reached maturity and was confounded by treatment switching, for which different crossover adjustment methods produced large variations in life years. Caregiver utilities were applied in the analysis, but there was a lack of guidance on their application and whether these were indicated in this appraisal. After consultation, the company updated the economic analysis with the MAIC. Incremental cost-effectiveness ratios comparing mogamulizumab against ECM were (depending on whether the HES or MAVORIC comparison were used) £31,030 or £32,634 per quality-adjusted life years (QALYs) gained according to the company’s base case and £38,274 or £80,555 per QALY gained according to the ERG’s base case. NICE did not recommend mogamulizumab for treating MF or SS in adults who have had at least one previous systemic treatment. This decision was subsequently appealed, and an appeal decision has been reached.