Multicriteria Decision Analysis to Support Health Technology Assessment Agencies: Benefits, Limitations, and the Way Forward

INTRODUCTION:Priority setting in health care has been long recognized as an intrinsically complex and value-laden process. Yet, Health Technology Assessment (HTA) agencies presently employ value assessment frameworks that are ill-fitted to capture the range and diversity of stakeholder values, and thereby risk to compromise the legitimacy of their recommendations. We propose ‘evidence-informed deliberative processes’ as an alternative framework with the aim to enhance this legitimacy.METHODS:The framework is based on an integration of two increasingly popular and complementary frameworks for priority setting: multi-criteria decision analysis (MCDA) and accountability for reasonableness (A4R), Evidence-informed deliberative processes are, on the one hand, based on early, continued stakeholder deliberation to learn about the importance of relevant social values. On the other hand, they are based on rational decision-making – through evidence-informed evaluation of the identified values.RESULTS:The framework has important implications for how HTA agencies should ideally organize their processes. Firstly, HTA agencies should take the responsibility to organize stakeholder involvement. Second, agencies are advised to integrate their assessment and appraisal phase, allowing for the timely collection of evidence on values that are considered relevant. Third, HTA agencies should subject their specification of decision-making criteria to public scrutiny. Fourth, agencies are advised to use a checklist of potentially relevant criteria, and to provide argumentation how each criterion affected the recommendation. Fifth, HTA agencies must publish their argumentation and install options for appeal.CONCLUSIONS:Adopting ‘evidence-informed deliberative processes’ as a value assessment framework could be an important step forward for HTA agencies to optimize the legitimacy of their priority setting decisions. Agencies can incorporate elements according to their needs and affordances.

The Use of MCDA in HTA: Great Potential, but More Effort Needed

The objectives of the study were to establish a benchmarking tool to collect metrics to enable increased clarity regarding the differences and similarities across health technology assessment (HTA) agencies, to assess performance within and across HTA agencies, identify areas in the HTA processes in which time is spent and to enable ongoing performance improvement. Common steps and milestones in the HTA process were identified for meaningful benchmarking among agencies. A benchmarking tool consisting of eighty-six questions providing information on HTA agency organizational aspects and information on individual new medicine review timelines and outcomes was developed with the input of HTA agencies and validated in a pilot study. Data on 109 HTA reviews from five HTA agencies were analyzed to demonstrate the utility of this tool. This study developed an HTA benchmarking methodology, comparative metrics showed considerable differences among the median timelines from assessment and appraisal to final HTA recommendation for the five agencies included in this analysis; these results were interpreted in conjunction with agency characteristics. It is feasible to find consensus among HTA agencies regarding the common milestones of the review process to map jurisdiction-specific processes against agreed metrics. Data on characteristics of agencies such as their scope and remit enabled results to be interpreted in the appropriate local context. This benchmarking tool has promising potential utility to improve the transparency of the review process and to facilitate both quality assurance and performance improvement in HTA agencies.

The purpose of this study was to systematically review the process for topic selection by health technology assessment (HTA) agencies around the world to provide the knowledge base for the improvement of topic selection frameworks in HTA agencies. A systematic search was conducted in PubMed and EMBASE to identify papers up to February 2019. Gray literature was identified by screening the Web sites of HTA agencies on the nonprofit member list of the International Network of Agencies for Health Technology Assessment (INAHTA). Data were extracted for each HTA agency and synthesized, with issues including general contextual information about each agency and the process of topic selection. Out of forty-nine nonprofit members of INAHTA, a total of seventeen HTA agencies with a framework for topic selection were identified from twenty-two included papers/documents. Multiple criteria were used for topic selection in all frameworks and agencies undertook multiple steps, which could include the specification of criteria for topic selection, identification of topics, short listing of potential topics, scoping of potential topics, scoring and ranking of potential topics, and deliberation and decision on final topics for HTA. Shortcomings were found in relation to methods of scoring and ranking as well as lack of monitoring and the evaluation of the process. Our study provides insights into the current practice of topic selection in HTA agencies. Multiple criteria decision analysis methodology appears highly relevant to these processes. A consensus approach for the development of methods of topic selection would be valuable for the HTA community.


 Several regulatory agencies have implemented processes to facilitate timely access to innovative therapies for patients with rare or life-threatening diseases with high unmet medical needs, typically with additional data-generation requirements to address important uncertainties in the existing evidence base at the time of regulatory approval. Many health technology assessment (HTA) agencies have adapted their processes to include time-limited recommendations (TLRs) for such therapies, which allows patients access to these medications while additional data are collected to verify the clinical efficacy, real-world effectiveness, or economic value of the new therapeutic. The objective of this Environmental Scan is to identify what HTA agencies have processes in place for TLRs and to describe these processes, if applicable.
 HTA agencies in North America, Europe, Australia, and New Zealand were considered for this report based on commonalities to the Canadian context in terms of geography and relevant regulatory or HTA and reimbursement processes. Seven HTA agencies (the National Institute for Health and Care Excellence, the Scottish Medicines Consortium, France’s Haute Autorité de Santé, the Dutch National Health Care Institute, the Italian Medicines Agency, the Belgium National Institute for Health and Disability Insurance, and the Australian Pharmaceutical Benefits Advisory Committee) that have implemented formal TLR processes, through managed entry agreements, special access funds, or other related programs were identified. Most TLR programs are referred to as managed access or entry agreements, but other types include interim acceptance, conditional inclusion, assessment contingent on additional evidence development, and additional specialized programs or pathways.
 TLRs are primarily used for promising (and sometimes high-cost) medicines intended to treat a condition with an important unmet medical need; some specify that the disease must be serious and life threatening (regardless of prevalence), others indicate that the process is used for orphan drugs intended to treat rare conditions, and others consider multiple criteria or do not have prerequisite criteria for eligible indications.
 Evidence generation requirements for TLRs vary across HTA agencies and are often dependent on the nature of the clinical and economic uncertainties identified in the initial assessment; sources include later follow-up from ongoing clinical trials, new clinical trials, real-world data, or a combination of both clinical and real-world studies. Additionally, most TLR processes have explicit maximum durations for the time-limited period, which range from 2 to 7 years across the included HTA agencies (although some allow for time adjustments or renewals); however, the French and Scottish HTA agencies determine the duration of the time-limited period on a case-by-case basis.
 Overall, this report provides an overview of HTA processes for TLRs across HTA agencies in Europe and Australia, many of which may be useful to inform the implementation of future TLR processes in Canada, as well as to refine existing ones.

PHP235 - INDIRECT TREATMENT COMPARISONS (ITC) FOR HTA IN EUROPE – UNDERSTANDING REQUIREMENTS FOR INDUSTRY SUBMISSIONS

In the drive towards faster patient access to treatments, health technology assessment (HTA) agencies are increasingly faced with reliance on evidence from surrogate endpoints, leading to increased decision uncertainty. This study undertook an updated survey of methodological guidance for using surrogate endpoints across international HTA agencies. We reviewed HTA and economic evaluation methods guidance from European, Australian and Canadian HTA agencies. We considered how guidelines addressed the methods for handling surrogate endpoints, including (1) level of evidence, (2) methods of validation, and (3) thresholds of acceptability. Across the 73 HTA agencies surveyed, 29 (40%) had methodological guidelines that made specific reference to consideration of surrogate outcomes. Of the 45 methods documents analysed, the majority [27 (60%)] were non-technology specific, 15 (33%) focused on pharmaceuticals and three (7%) on medical devices. The principles of the European network for Health Technology Assessment (EUnetHTA) guidelines published in 2015 on the handling of surrogate endpoints appear to have been adopted by many European HTA agencies, i.e. preference for final patient-relevant outcomes and reliance on surrogate endpoints with biological plausibility and epidemiological evidence of the association between the surrogate and final endpoint. Only a small number of HTA agencies (UK National Institute for Care and Excellence; the German Institute for Medical Documentation and Information and Institute for Quality and Efficiency in Health Care; the Australian Pharmaceutical Benefits Advisory Committee; and the Canadian Agency for Drugs and Technologies in Health) have developed more detailed prescriptive criteria for the acceptance of surrogate endpoints, e.g. meta-analyses of randomised controlled trials showing strong association between the treatment effect on the surrogate and final outcomes. As the decision uncertainty associated with reliance on surrogate endpoints carries a risk to patients and society, there is a need for HTA agencies to develop more detailed methodological guidance for consistent selection and evaluation of health technologies that lack definitive final patient-relevant outcome evidence at the time of the assessment.

Recommendations by countries' health technology assessment (HTA) agencies are used to decide which new therapies warrant the allocation of limited health-care resources to make them available through publicly funded health systems. This process is of public health importance for balancing the dual aims of optimising patient outcomes while ensuring financial sustainability. We evaluated which factors affect HTA outcomes and the time to positive HTA outcome, focussing on the role of clinical benefit evaluated with the European Society for Medical Oncology-Magnitude of Clinical Benefit Scale (ESMO-MCBS). In this retrospective analysis, data were extracted from publicly available HTA reports and related sources from six country settings and their respective HTA agencies (Australia, Canada, England, France, the Canadian province of Quebec, and Scotland). We evaluated new cancer medicines for treating solid tumours in a non-curative setting with published ESMO-MCBS scores and that had been assessed by at least three HTA agencies between Jan 1, 2011, and Dec 31, 2020. Using ESMO-MCBS score as an independent variable, we did descriptive and multivariable regression analyses to evaluate: (1) factors associated with the time between marketing authorisation and positive (unrestricted [List] and restricted [List with Constraints]) HTA outcome; and (2) factors associated with HTA outcomes. 67 medicine-indication pairs used in non-curative settings were identified, totalling 360 HTA submissions (medicine-indication-country triplets) reviewed by the six HTA agencies. Factors significantly associated with a reduced interval between marketing authorisation and a positive (unrestricted or restricted) HTA outcome included a high ESMO-MCBS score (ie, 4 or 5, vs a low or average score of 1-3; hazard ratio [HR] per 1 month increment 1·42 [95% CI 1·11-1·81], p=0·0055), parallel review (vs standard marketing authorisation process; HR 1·69 [1·13-2·54], p=0·011), having a risk-sharing agreement or special funding arrangements (vs no funding agreement, HR 4·62 [95% CI 2·51-8·51], p<0·0001, and HR 4·16 [2·03-8·50], p=0·0001, respectively), and assessment by particular HTA agencies (pan-Canadian Oncology Drug Review vs National Institute for Health and Care Excellence [NICE], HR 2·82 [1·68-4·75], p=0·0001; and Haute Autorité de Santé vs NICE, HR 5·70 [2·87-11·33], p<0·0001). Accelerated marketing authorisation was significantly associated with a longer time to positive HTA outcome (vs standard authorisation process; HR 0·70 [95% CI 0·51-0·95], p=0·024). Positive HTA outcomes (both unrestricted and restricted) were significantly associated with a high ESMO-MCBS score (vs low or average ESMO-MCBS score; relative risk ratio [RRR] 14·10 [95% CI 3·54-56·20], p=0·0002, and RRR 4·52 [1·90-10·75], p=0·0006, respectively) and acknowledgment of unmet medical need (vs unmet need not recorded, RRR 22·73 [5·51-93·73], p<0·0001, and RRR 1·87 [1·18-2·97], p=0·0075, respectively). By contrast, positive HTA outcomes (unrestricted and restricted) were inversely associated with uncertainties regarding inputs to economic models informing HTA submissions (vs uncertainties not recorded, RRR 0·28 [0·10-0·78], p=0·014, and RRR 0·45 [0·25-0·82], p=0·010, respectively). Regarding country-relevant effects, inverse associations with positive HTA outcomes (both unrestricted and restricted) were observed for assessment in Quebec (vs England; RRR 1·15×10-6 [1·44×10-7-9·09×10-6], p<0·0001, and RRR 0·33 (0·24-0·46), p<0·0001, respectively) and for assessment in Australia (vs England; RRR 1·78×10-6 [1·04×10-8-3·00×10-4], p<0·0001, and RRR 0·30 [0·15-0·61], p=0·0008, respectively). Several factors informed HTA outcomes for new cancer medicines. A high ESMO-MCBS score, defined as indicating substantial clinical benefit, increased the likelihood of a positive HTA outcome and shortened the interval between marketing authorisation and HTA outcome, and this association was not affected by other variables. Additional factors informing HTA outcomes include evidence uncertainties and unmet medical need. Country-relevant differences exist in the time-to-HTA outcome and the propensity of some countries to achieve positive (restricted or unrestricted) outcomes compared with others. None.

Policies for Use of Real-World Data in Health Technology Assessment (HTA): A Comparative Study of Six HTA Agencies.

BackgroundSurvival extrapolation is used to predict patients' overall survival beyond clinical trial follow-up. It can significantly affect the results of a cost-effectiveness analysis and subsequent pricing and reimbursement decisions. However, selecting an appropriate model involves subjectivity, leading to discrepancies between methods submitted by manufacturers and those accepted by health technology assessment (HTA) agencies. This review describes the acceptance and criticisms of overall survival extrapolation methods by HTA agencies in England, France, and Australia.MethodsElectronic searches conducted on September 4, 2022, identified HTA evaluations for oncology therapies indicated for the treatment of solid tumors from the National Institute for Health and Care Excellence (NICE) in England, the Haute Autorité de Santé (HAS) in France, and the Pharmaceutical Benefits Advisory Committee (PBAC) in Australia, published between August 2017 and August 2022. Information on the overall survival extrapolation model submitted by the manufacturer was extracted. The acceptance decision of the HTA agency and the key criticisms were also recorded.ResultsA total of 140 HTA evaluations were identified. The initial overall survival extrapolation method was accepted in 21% of cases. The most frequently cited criticisms related to a lack of or inappropriate incorporation of treatment effect waning over time (31%). Other criticisms included choice of parametric distribution, in which multiple distributions were often considered plausible (24%); immaturity of survival data (15%); and concerns about the proportional hazards assumption, which lacked clinical validity (8%).ConclusionThis review highlights the low acceptance of survival extrapolation methods and the areas of discordance between manufacturers and HTA agencies in England, France, and Australia. Low acceptance rates of survival extrapolation methods by HTA bodies can affect pricing and reimbursement decisions of new therapies, delaying patient access.HighlightsThis review found that the survival extrapolation methods initially submitted by companies were accepted in only 21% of cases.The most common areas of discordance between companies and HTA agencies were inappropriate modeling of treatment effect over time, choice of parametric distribution, immaturity of survival data, and concerns about the proportional hazards assumption.There is a need for more consistent guidance on the selection of an appropriate extrapolation method to limit the inherent subjectivity surrounding survival curve selection.

Background: Evidence-informed deliberative processes (EDPs) were recently introduced to guide health technology assessment (HTA) agencies to improve their processes towards more legitimate decision-making. The EDP framework provides guidance that covers the HTA process, ie, contextual factors, installation of an appraisal committee, selecting health technologies and criteria, assessment, appraisal, and communication and appeal. The aims of this study were to identify the level of use of EDPs by HTA agencies, identify their needs for guidance, and to learn about best practices.Methods: A questionnaire for an online survey was developed based on the EDP framework, consisting of elements that reflect each part of the framework. The survey was sent to members of the International Network of Agencies for Health Technology Assessment (INAHTA). Two weeks following the invitation, a reminder was sent. The data collection took place between September-December 2018. Results: Contact persons from 27 member agencies filled out the survey (response rate: 54%), of which 25 completed all questions. We found that contextual factors to support HTA development and the critical elements regarding conducting and reporting on HTA are overall in place. Respondents indicated that guidance was needed for specific elements related to selecting technologies and criteria, appraisal, and communication and appeal. With regard to best practices, the Canadian Agency for Drugs and Technologies and the National Institute for Health and Care Excellence (NICE, UK) were most often mentioned. Conclusion: This is the first survey among HTA agencies regarding the use of EDPs and provides useful information for further developing a practical guide for HTA agencies around the globe. The results could support HTA agencies in improving their processes towards more legitimate decision-making, as they could serve as a baseline measurement for future monitoring and evaluation.

Standard outcome sets developed by the International Consortium for Health Outcomes Measurement (ICHOM) facilitate value‐based health care in healthcare practice and have gained traction from regulators and Health Technology Assessment (HTA) agencies that regularly assess the value of new medicines. We aimed to assess the extent to which the outcomes used by regulators and HTA agencies are patient‐relevant, by comparing these to ICHOM standard sets. We conducted a cross‐sectional comparative analysis of ICHOM standard sets, and publicly available regulatory and HTA assessment guidelines. We focused on oncology due to many new medicines being developed, which are accompanied by substantial uncertainty regarding the relevance of these treatments for patients. A comparison of regulatory and HTA assessment guidelines, and ICHOM standard sets showed that both ICHOM and regulators stress the importance of disease‐specific outcomes. On the other hand, HTA agencies have a stronger focus on generic outcomes in order to allow comparisons across disease areas. Overall, similar outcomes are relevant for market access, reimbursement, and in ICHOM standard sets. However, some differences are apparent, such as the acceptability of intermediate outcomes. These are recommended in ICHOM standard sets, but regulators are more likely to accept intermediate outcomes than HTA agencies. A greater level of alignment in outcomes accepted may enhance the efficiency of regulatory and HTA processes, and increase timely access to new medicines. ICHOM standard sets may help align these outcomes. However, some differences in outcomes used may remain due to the different purposes of regulatory and HTA decision‐making.

PSY188 - USE OF MANAGED ENTRY AGREEMENTS IN INFLAMMATORY DISORDERS AMONG KEY HEALTH TECHNOLOGY ASSESSMENT AGENCIES

The aim of the study is to make an international comparison of Health Technology Assessment (HTA) Agencies, to show their similarities and differences. An e-mail questionnaire was sent to thirty HTA agencies internationally. Questions related to the structure of the agency, the relationship with health-related institutions, the prescriptiveness of the decisions taken, the main core and the modalities to spread the assessment, and the type of funding. Twenty-four HTA Agencies answered the questionnaire: 25 percent in America, 4.2 percent in Australia, and 70.8 percent in Europe. Fifty-four percent of HTA Agencies are governmental institutions (83.3 percent have central government funding), while 62.5 percent have relationships with health-related governmental institutions. Of the agencies, 87 percent reported that their decisions are not prescriptive, while for 20.8 percent and 8.3 percent of them stated that this was the case totally or partially, respectively, especially for the governmental and American Agencies. Seventeen agencies (70.8 percent) declared their work on multiannual programs (77 percent of the governmental HTA Agencies and 100 percent of the American ones). The assessments mainly addressed diagnostic procedures (85.7 percent) and pharmaceuticals (25 percent). The most common way to disseminate results is by means of paper report (91.7 percent), followed by the Internet (16.7 percent), and seminars to expert audiences (12.5 percent). The comparative analysis of HTA Agencies showed that governmental and American Agencies have a profound impact on the prescriptiveness of their assessment, and this could be linked to the fact that these types of Agencies work on multiannual programs. European and American HTA Agencies have many similarities in terms of type of assessment, funding, and dissemination of results.

A central function of health technology assessment (HTA) agencies is the production of HTA reports to support evidence-informed policy and decision making. HTA agencies are interested in understanding the mechanisms of HTA impact, which can be understood as the influence or impact of HTA report findings on decision making at various levels of the health system. The members of the International Network of Agencies for HTA (INAHTA) meet at their annual Congress where impact story sharing is one important activity. This paper summarizes four stories of HTA impact that were finalists for the David Hailey Award for Best Impact Story.The methods to measure impact include: document review; claims analysis and review of reimbursement status; citation analysis; qualitative evaluation of stakeholders' views; and review of media response. HTA agency staff also observed changes in government activities and priorities based on the HTA. Impact assessment can provide information to improve the HTA process, for example, the value of patient and clinician engagement in the HTA process to better define the assessment question and literature reviews in a more holistic and balanced way.HTA reports produced by publicly funded HTA agencies are valued by health systems around the globe as they support decision making regarding the appropriate use, pricing, reimbursement, and disinvestment of health technologies. HTAs can also have a positive impact on information sharing between different levels of government and across stakeholder groups. These stories show how HTA can have a significant impact, irrespective of the health system and health technology being assessed.