Systematic review protocol on evidence synthesis for health technology decisions in ultra-rare diseases.

Artificial Intelligence in Health Technology Assessment Submissions: A Targeted Review of Global Policy and Practice.


 Several regulatory agencies have implemented processes to facilitate timely access to innovative therapies for patients with rare or life-threatening diseases with high unmet medical needs, typically with additional data-generation requirements to address important uncertainties in the existing evidence base at the time of regulatory approval. Many health technology assessment (HTA) agencies have adapted their processes to include time-limited recommendations (TLRs) for such therapies, which allows patients access to these medications while additional data are collected to verify the clinical efficacy, real-world effectiveness, or economic value of the new therapeutic. The objective of this Environmental Scan is to identify what HTA agencies have processes in place for TLRs and to describe these processes, if applicable.
 HTA agencies in North America, Europe, Australia, and New Zealand were considered for this report based on commonalities to the Canadian context in terms of geography and relevant regulatory or HTA and reimbursement processes. Seven HTA agencies (the National Institute for Health and Care Excellence, the Scottish Medicines Consortium, France’s Haute Autorité de Santé, the Dutch National Health Care Institute, the Italian Medicines Agency, the Belgium National Institute for Health and Disability Insurance, and the Australian Pharmaceutical Benefits Advisory Committee) that have implemented formal TLR processes, through managed entry agreements, special access funds, or other related programs were identified. Most TLR programs are referred to as managed access or entry agreements, but other types include interim acceptance, conditional inclusion, assessment contingent on additional evidence development, and additional specialized programs or pathways.
 TLRs are primarily used for promising (and sometimes high-cost) medicines intended to treat a condition with an important unmet medical need; some specify that the disease must be serious and life threatening (regardless of prevalence), others indicate that the process is used for orphan drugs intended to treat rare conditions, and others consider multiple criteria or do not have prerequisite criteria for eligible indications.
 Evidence generation requirements for TLRs vary across HTA agencies and are often dependent on the nature of the clinical and economic uncertainties identified in the initial assessment; sources include later follow-up from ongoing clinical trials, new clinical trials, real-world data, or a combination of both clinical and real-world studies. Additionally, most TLR processes have explicit maximum durations for the time-limited period, which range from 2 to 7 years across the included HTA agencies (although some allow for time adjustments or renewals); however, the French and Scottish HTA agencies determine the duration of the time-limited period on a case-by-case basis.
 Overall, this report provides an overview of HTA processes for TLRs across HTA agencies in Europe and Australia, many of which may be useful to inform the implementation of future TLR processes in Canada, as well as to refine existing ones.

Background: Evidence-informed deliberative processes (EDPs) were recently introduced to guide health technology assessment (HTA) agencies to improve their processes towards more legitimate decision-making. The EDP framework provides guidance that covers the HTA process, ie, contextual factors, installation of an appraisal committee, selecting health technologies and criteria, assessment, appraisal, and communication and appeal. The aims of this study were to identify the level of use of EDPs by HTA agencies, identify their needs for guidance, and to learn about best practices.Methods: A questionnaire for an online survey was developed based on the EDP framework, consisting of elements that reflect each part of the framework. The survey was sent to members of the International Network of Agencies for Health Technology Assessment (INAHTA). Two weeks following the invitation, a reminder was sent. The data collection took place between September-December 2018. Results: Contact persons from 27 member agencies filled out the survey (response rate: 54%), of which 25 completed all questions. We found that contextual factors to support HTA development and the critical elements regarding conducting and reporting on HTA are overall in place. Respondents indicated that guidance was needed for specific elements related to selecting technologies and criteria, appraisal, and communication and appeal. With regard to best practices, the Canadian Agency for Drugs and Technologies and the National Institute for Health and Care Excellence (NICE, UK) were most often mentioned. Conclusion: This is the first survey among HTA agencies regarding the use of EDPs and provides useful information for further developing a practical guide for HTA agencies around the globe. The results could support HTA agencies in improving their processes towards more legitimate decision-making, as they could serve as a baseline measurement for future monitoring and evaluation.

In the drive towards faster patient access to treatments, health technology assessment (HTA) agencies are increasingly faced with reliance on evidence from surrogate endpoints, leading to increased decision uncertainty. This study undertook an updated survey of methodological guidance for using surrogate endpoints across international HTA agencies. We reviewed HTA and economic evaluation methods guidance from European, Australian and Canadian HTA agencies. We considered how guidelines addressed the methods for handling surrogate endpoints, including (1) level of evidence, (2) methods of validation, and (3) thresholds of acceptability. Across the 73 HTA agencies surveyed, 29 (40%) had methodological guidelines that made specific reference to consideration of surrogate outcomes. Of the 45 methods documents analysed, the majority [27 (60%)] were non-technology specific, 15 (33%) focused on pharmaceuticals and three (7%) on medical devices. The principles of the European network for Health Technology Assessment (EUnetHTA) guidelines published in 2015 on the handling of surrogate endpoints appear to have been adopted by many European HTA agencies, i.e. preference for final patient-relevant outcomes and reliance on surrogate endpoints with biological plausibility and epidemiological evidence of the association between the surrogate and final endpoint. Only a small number of HTA agencies (UK National Institute for Care and Excellence; the German Institute for Medical Documentation and Information and Institute for Quality and Efficiency in Health Care; the Australian Pharmaceutical Benefits Advisory Committee; and the Canadian Agency for Drugs and Technologies in Health) have developed more detailed prescriptive criteria for the acceptance of surrogate endpoints, e.g. meta-analyses of randomised controlled trials showing strong association between the treatment effect on the surrogate and final outcomes. As the decision uncertainty associated with reliance on surrogate endpoints carries a risk to patients and society, there is a need for HTA agencies to develop more detailed methodological guidance for consistent selection and evaluation of health technologies that lack definitive final patient-relevant outcome evidence at the time of the assessment.

The purpose of this study was to systematically review the process for topic selection by health technology assessment (HTA) agencies around the world to provide the knowledge base for the improvement of topic selection frameworks in HTA agencies. A systematic search was conducted in PubMed and EMBASE to identify papers up to February 2019. Gray literature was identified by screening the Web sites of HTA agencies on the nonprofit member list of the International Network of Agencies for Health Technology Assessment (INAHTA). Data were extracted for each HTA agency and synthesized, with issues including general contextual information about each agency and the process of topic selection. Out of forty-nine nonprofit members of INAHTA, a total of seventeen HTA agencies with a framework for topic selection were identified from twenty-two included papers/documents. Multiple criteria were used for topic selection in all frameworks and agencies undertook multiple steps, which could include the specification of criteria for topic selection, identification of topics, short listing of potential topics, scoping of potential topics, scoring and ranking of potential topics, and deliberation and decision on final topics for HTA. Shortcomings were found in relation to methods of scoring and ranking as well as lack of monitoring and the evaluation of the process. Our study provides insights into the current practice of topic selection in HTA agencies. Multiple criteria decision analysis methodology appears highly relevant to these processes. A consensus approach for the development of methods of topic selection would be valuable for the HTA community.

PRO89 TOO ULTRA-RARE FOR CARE? ORPHAN DRUG AVAILABILITY FOR RESPIRATORY DISEASES: A SYSTEMATIC REVIEW

The objectives of the study were to establish a benchmarking tool to collect metrics to enable increased clarity regarding the differences and similarities across health technology assessment (HTA) agencies, to assess performance within and across HTA agencies, identify areas in the HTA processes in which time is spent and to enable ongoing performance improvement. Common steps and milestones in the HTA process were identified for meaningful benchmarking among agencies. A benchmarking tool consisting of eighty-six questions providing information on HTA agency organizational aspects and information on individual new medicine review timelines and outcomes was developed with the input of HTA agencies and validated in a pilot study. Data on 109 HTA reviews from five HTA agencies were analyzed to demonstrate the utility of this tool. This study developed an HTA benchmarking methodology, comparative metrics showed considerable differences among the median timelines from assessment and appraisal to final HTA recommendation for the five agencies included in this analysis; these results were interpreted in conjunction with agency characteristics. It is feasible to find consensus among HTA agencies regarding the common milestones of the review process to map jurisdiction-specific processes against agreed metrics. Data on characteristics of agencies such as their scope and remit enabled results to be interpreted in the appropriate local context. This benchmarking tool has promising potential utility to improve the transparency of the review process and to facilitate both quality assurance and performance improvement in HTA agencies.

BackgroundSurvival extrapolation is used to predict patients' overall survival beyond clinical trial follow-up. It can significantly affect the results of a cost-effectiveness analysis and subsequent pricing and reimbursement decisions. However, selecting an appropriate model involves subjectivity, leading to discrepancies between methods submitted by manufacturers and those accepted by health technology assessment (HTA) agencies. This review describes the acceptance and criticisms of overall survival extrapolation methods by HTA agencies in England, France, and Australia.MethodsElectronic searches conducted on September 4, 2022, identified HTA evaluations for oncology therapies indicated for the treatment of solid tumors from the National Institute for Health and Care Excellence (NICE) in England, the Haute Autorité de Santé (HAS) in France, and the Pharmaceutical Benefits Advisory Committee (PBAC) in Australia, published between August 2017 and August 2022. Information on the overall survival extrapolation model submitted by the manufacturer was extracted. The acceptance decision of the HTA agency and the key criticisms were also recorded.ResultsA total of 140 HTA evaluations were identified. The initial overall survival extrapolation method was accepted in 21% of cases. The most frequently cited criticisms related to a lack of or inappropriate incorporation of treatment effect waning over time (31%). Other criticisms included choice of parametric distribution, in which multiple distributions were often considered plausible (24%); immaturity of survival data (15%); and concerns about the proportional hazards assumption, which lacked clinical validity (8%).ConclusionThis review highlights the low acceptance of survival extrapolation methods and the areas of discordance between manufacturers and HTA agencies in England, France, and Australia. Low acceptance rates of survival extrapolation methods by HTA bodies can affect pricing and reimbursement decisions of new therapies, delaying patient access.HighlightsThis review found that the survival extrapolation methods initially submitted by companies were accepted in only 21% of cases.The most common areas of discordance between companies and HTA agencies were inappropriate modeling of treatment effect over time, choice of parametric distribution, immaturity of survival data, and concerns about the proportional hazards assumption.There is a need for more consistent guidance on the selection of an appropriate extrapolation method to limit the inherent subjectivity surrounding survival curve selection.

Plain English summaryA number of health technology assessment (HTA) organisations have developed processes to engage patients in the assessment of new health technologies such as pharmaceuticals, diagnostic tests, devices or medical procedures. Typically, this involves the HTA agency providing an opportunity for patient advocates and their patient organisations (support groups for patients with a specific disease or condition) to provide submissions detailing experiences with the disease and the health technology that is being assessed. While some literature exists about how HTA agencies view the engagement of patients in the HTA process, it is not yet clear how the patient advocates and patient organisations themselves view this engagement. To answer this question, we surveyed the views of patient advocates who were members of patient organisations known to be engaged in the process of HTA or evidence-based practice. Snowballing – that is, passing on the survey invitation from individuals invited to take part in the survey to other individuals – occurred in one of the countries. The responses in this country provided a very useful comparison between the views of people who were appointed as the ‘patient representatives’ on an HTA committee with those who contributed input as part of the general patient organisation engagement process. Our findings identify gaps in understanding of the purpose of patient involvement and whether patient organisations felt their input made a difference, the information and support provided, and if and how feedback is given to the patient organisations. Our work can help inform further research as well as continuing improvements in HTA patient engagement processes.BackgroundPatient involvement in health technology assessment (HTA) processes is becoming more frequent. However, it is not clear how patient advocates and their disease-based patient organisations that are involved in HTA view their involvement. We report on the results of an international survey of patient advocates and members of patient organisations about their experiences and perceptions of that involvement.MethodsA 16-question survey was sent out to patient advocates and members of patient groups known to be involved in HTA processes or evidence-based practice. The survey consisted of open-ended questions focusing on respondent characteristics, stage and nature of involvement, support from HTA agencies for involvement, purpose of involvement, feedback on involvement, and whether the respondents felt that their input made a difference.ResultsOf 16 individuals who received the survey, 15 responded. Three, from Italy, Israel and Japan, were not involved in HTA in their country. Respondents from the following countries reported involvement in HTA processes: Canada, England, Scotland, and Wales, The Netherlands, Australia, Taiwan. The respondents indicated that HTA agencies reach out to them either actively or passively, and that their involvement is often at the appraisal stage of HTA. Typically, they reported involvement as either participants in committees or providers of submissions to HTA agencies. A wide range of approaches to supporting patient involvement by the HTA agencies was identified by respondents – including personal and telephone support, online resources, training and provision of information – but the level and type of support reported was uneven across jurisdictions. Not all respondents were clear on the purpose of their involvement in HTA, although some were able to cite specific examples of how their input made a difference; members of an HTA decision-making committee appeared to have a better understanding and were able to give examples. Feedback from HTA agencies to the patient groups on their submissions is often not provided.ConclusionsAlthough considerable progress has been made in terms of engaging patients and patient groups in HTA, gaps remain in how involvement is supported, including facilitating involvement, clarity on roles, two-way flow of information, and methods for enhancing communication between patient organisations and HTA agencies.

IntroductionOver the past decade there have been increasing interactions between health technology assessment (HTA) agencies through international networks at the policy level and European joint actions at the product level. A pilot project is underway to explore collaboration beyond Europe between HTA agencies in Australia, Canada, and the UK. This study the compared HTA recommendations of new active substances (NAS) appraised by Australia’s Pharmaceutical Benefits Advisory Committee (PBAC), Canada’s CADTH, and England’s National Institute for Health and Care Excellence (NICE).MethodsUsing publicly available data and established benchmarking methodology, we examined 45 NAS appraised by PBAC, CADTH, and NICE between 2017 and 2021. Analysis was performed to assess rollout time from regulatory to HTA recommendation, and to the first HTA recommendation.ResultsMost products were submitted to the Europe Medicine Agency first (89%). However, 71 percent of NAS in Australia and 69 percent in Canada were submitted to HTA in parallel with regulatory review, which shortened overall rollout time. The median HTA submission gap among the three agencies was 140 days in Australia, 102 days in Canada, and 8 days in England. PBAC had the highest number of negative recommendations (51%), followed by CADTH (18%), and NICE (5%). The congruence of HTA decisions was highest between CADTH and NICE (56%), compared with PBAC and CADTH (11%), and PBAC and NICE (20%)ConclusionsTo achieve a more collaborative HTA process it is necessary to understand the rollout time in these jurisdictions. This study identified submission gaps among the regulatory agencies, but there may be more synergy in the future as regulators in the three jurisdictions work collaboratively through the Access Consortium. Currently, the submission gap for the three HTA agencies was mostly within six months, making collaboration on joint assessment possible. We observed divergences in HTA recommendations due to the methodology and decision criteria applied by each agency. Therefore, collaboration on assessment should build on the clinical aspects, although HTA decisions should be grounded in the local context.

A central function of health technology assessment (HTA) agencies is the production of HTA reports to support evidence-informed policy and decision making. HTA agencies are interested in understanding the mechanisms of HTA impact, which can be understood as the influence or impact of HTA report findings on decision making at various levels of the health system. The members of the International Network of Agencies for HTA (INAHTA) meet at their annual Congress where impact story sharing is one important activity. This paper summarizes four stories of HTA impact that were finalists for the David Hailey Award for Best Impact Story.The methods to measure impact include: document review; claims analysis and review of reimbursement status; citation analysis; qualitative evaluation of stakeholders' views; and review of media response. HTA agency staff also observed changes in government activities and priorities based on the HTA. Impact assessment can provide information to improve the HTA process, for example, the value of patient and clinician engagement in the HTA process to better define the assessment question and literature reviews in a more holistic and balanced way.HTA reports produced by publicly funded HTA agencies are valued by health systems around the globe as they support decision making regarding the appropriate use, pricing, reimbursement, and disinvestment of health technologies. HTAs can also have a positive impact on information sharing between different levels of government and across stakeholder groups. These stories show how HTA can have a significant impact, irrespective of the health system and health technology being assessed.

Policies for Use of Real-World Data in Health Technology Assessment (HTA): A Comparative Study of Six HTA Agencies.

IntroductionIncorporating technologies for ultrarare diseases (URD) poses challenges for global health technology assessment (HTA) agencies. Difficulties include defining an analytical framework and establishing differentiated cost-effectiveness thresholds. The rise of technological innovations intensifies demands from healthcare professionals, media, and the general population, placing pressure on healthcare systems in developing countries.MethodsTo analyze ultrarare medicine costs in submissions to the Brazilian National Committee for Health Technology Incorporation (CONITEC), data from HTA reports on URD (from 2012 to 2022) were extracted. Diseases were classified as URD based on an epidemiological criterion or Orphanet consultation (prevalence ≤1 per 50,000 inhabitants). Extracted variables included initial and final prices, annual patient cost, incremental cost-effectiveness ratio (ICER), and initial and final CONITEC recommendations. Price differences were calculated by the Brazilian Medicines Market Regulation Chamber.ResultsAmong 53 reports, 30 featured economic evaluations, with only 13.3 percent initially receiving positive recommendations. However, eight gained favor, including post-consultation, price-conditioned, and risk sharing-based approvals. Annual medication costs ranged from USD17,439.20 to USD1,108,237.00 per patient, with discounts of between 25 and 64 percent. Despite some technologies having ICERs that were significantly higher than the national threshold, no statistical relationship was found between ICERs and recommendations. Monthly and annual costs varied, with higher costs for heterogeneous diseases and lower costs for metabolic conditions. Sensitivity analyses, using both deterministic and probabilistic analyses, were conducted in 58 percent of the reports.ConclusionsIncorporation of technologies for URD does not correlate with lower annual costs or increased discounts because costs are not considered in isolation by CONITEC’s decision-making process. Recognizing URD as a subgroup with distinct criteria may enhance the implementation of HTA processes tailored to the unique challenges of these conditions.

INTRODUCTION:Priority setting in health care has been long recognized as an intrinsically complex and value-laden process. Yet, Health Technology Assessment (HTA) agencies presently employ value assessment frameworks that are ill-fitted to capture the range and diversity of stakeholder values, and thereby risk to compromise the legitimacy of their recommendations. We propose ‘evidence-informed deliberative processes’ as an alternative framework with the aim to enhance this legitimacy.METHODS:The framework is based on an integration of two increasingly popular and complementary frameworks for priority setting: multi-criteria decision analysis (MCDA) and accountability for reasonableness (A4R), Evidence-informed deliberative processes are, on the one hand, based on early, continued stakeholder deliberation to learn about the importance of relevant social values. On the other hand, they are based on rational decision-making – through evidence-informed evaluation of the identified values.RESULTS:The framework has important implications for how HTA agencies should ideally organize their processes. Firstly, HTA agencies should take the responsibility to organize stakeholder involvement. Second, agencies are advised to integrate their assessment and appraisal phase, allowing for the timely collection of evidence on values that are considered relevant. Third, HTA agencies should subject their specification of decision-making criteria to public scrutiny. Fourth, agencies are advised to use a checklist of potentially relevant criteria, and to provide argumentation how each criterion affected the recommendation. Fifth, HTA agencies must publish their argumentation and install options for appeal.CONCLUSIONS:Adopting ‘evidence-informed deliberative processes’ as a value assessment framework could be an important step forward for HTA agencies to optimize the legitimacy of their priority setting decisions. Agencies can incorporate elements according to their needs and affordances.

Health technology assessment (HTA) agencies assess evidence to support decision making about which technologies to provide and pay for in the health system. HTA impact is understood as the influence that HTA report findings can have in the health system, including impacts on reimbursement decisions, changes to health outcomes, or broader system or societal impacts. The International Network of Agencies for Health Technology Assessment (INAHTA) is a global network of publicly funded HTA agencies. INAHTA's mission, in part, is to advance the impact of HTA to support reimbursement decisions and the optimal use of health system resources. Each year, INAHTA awards the David Hailey Award for Best Impact Story to the member agency that shares the best story, as voted by fellow members, about HTA impact. The impact story sharing program in INAHTA contributes to a deeper understanding of what works well (or not so well) in achieving HTA impact. This paper provides six impact stories from agencies that were finalists for the 2021 and 2022 David Hailey Impact Award for Best Impact Story: the Institut national d'excellence en santé et en services sociaux, the Malaysian Health Technology Assessment Section, Ontario Health, the Center for Drug Evaluation, the National Institute for Health and Care Excellence, and Health Technology Wales. These stories demonstrate that HTA agencies can, in differing ways, effectively support governments in their efforts to place evidence at the centre of decision making.