Relative Effectiveness Assessment Research Articles

Uncertainty in Long-Term Relative Effectiveness of Medicines in Health Technology Assessment

ObjectivesUncertainty regarding the long-term relative effectiveness is an important factor in health technology assessment (HTA) of medicines. This study investigated how different HTA bodies address this uncertainty in their assessments. MethodsA total of 49 HTA reports from 6 national HTA bodies, assessing 9 medicines for spinal muscular atrophy, cystic fibrosis, and hypercholesterolemia, were included. In these reports, 81 relative effectiveness assessments and 45 cost-effectiveness assessments were performed on an indication level. We collected information on included trials, assessment outcomes, uncertainty regarding the long-term effectiveness, proposed managed entry agreements, and reassessments. ResultsUncertainty regarding the long-term effectiveness was an important consideration in almost all cost-effectiveness assessments (91%) and three-quarters of relative effectiveness assessments (74%), despite differences in methodologies among HTA bodies. There were considerable differences in the amount and type of long-term effectiveness data included by HTA bodies due to timing and inclusion criteria. In total 23 managed entry agreements were proposed of which 14 were linked to uncertainty regarding the long-term effectiveness. In addition, 13 reassessments were performed of which 4 led to an increase in patient access because of more available long-term effectiveness data. ConclusionsUncertainty regarding the long-term effectiveness is an important challenge for HTA bodies. There are large differences in the acceptance of evidence among HTA bodies, which leads to heterogeneity in the inclusion of available long-term effectiveness data for decision making. In cases with large uncertainty regarding the long-term effectiveness, outcome-based agreements and reassessments are used by HTA bodies, but differently between HTA bodies and indications.

Integrating microdosimetricin vitroRBE models for particle therapy into TOPAS MC using the MicrOdosimetry-based modeliNg for RBE ASsessment (MONAS) tool.

Objective.In this paper, we present MONAS (MicrOdosimetry-based modelliNg for relative biological effectiveness (RBE) ASsessment) toolkit. MONAS is a TOPAS Monte Carlo extension, that combines simulations of microdosimetric distributions with radiobiological microdosimetry-based models for predicting cell survival curves and dose-dependent RBE.Approach.MONAS expands TOPAS microdosimetric extension, by including novel specific energy scorers to calculate the single- and multi-event specific energy microdosimetric distributions at different micrometer scales. These spectra are used as physical input to three different formulations of themicrodosimetric kinetic model, and to thegeneralized stochastic microdosimetric model(GSM2), to predict dose-dependent cell survival fraction and RBE. MONAS predictions are then validated against experimental microdosimetric spectra andin vitrosurvival fraction data. To show the MONAS features, we present two different applications of the code: (i) the depth-RBE curve calculation from a passively scattered proton SOBP and monoenergetic12C-ion beam by using experimentally validated spectra as physical input, and (ii) the calculation of the 3D RBE distribution on a real head and neck patient geometry treated with protons.Main results.MONAS can estimate dose-dependent RBE and cell survival curves from experimentally validated microdosimetric spectra with four clinically relevant radiobiological models. From the radiobiological characterization of a proton SOBP and12C fields, we observe the well-known trend of increasing RBE values at the distal edge of the radiation field. The 3D RBE map calculated confirmed the trend observed in the analysis of the SOBP, with the highest RBE values found in the distal edge of the target.Significance.MONAS extension offers a comprehensive microdosimetry-based framework for assessing the biological effects of particle radiation in both research and clinical environments, pushing closer the experimental physics-based description to the biological damage assessment, contributing to bridging the gap between a microdosimetric description of the radiation field and its application in proton therapy treatment with variable RBE.

PT29 Predicting PICOs for EU Joint Clinical Assessment: Lessons from PICOs in Relative Effectiveness Assessments (REA) from EUnetHTA Joint Action 3 Project

PT29 Predicting PICOs for EU Joint Clinical Assessment: Lessons from PICOs in Relative Effectiveness Assessments (REA) from EUnetHTA Joint Action 3 Project

Differences in evidentiary requirements for oncology drug effectiveness assessments among six European health technology assessment bodies — can alignment be improved?

ABSTRACT Objectives Evidentiary requirements for relative effectiveness assessment vary among European health technology assessment (HTA) bodies, affecting the time to HTA decision-making and potentially delaying time to patient access. Improved alignment may reduce this time; therefore, we aim to analyze the differences in evidentiary requirements for oncology drug assessments among European HTA bodies and provide recommendations toward an increased alignment. Methods Interviews were conducted with stakeholders in drug assessments of Italy, the Netherlands, Poland, Portugal, England and Wales, and Sweden about evidentiary requirements for several subdomains to identify differences and obtain recommendations for addressing differences. The interview results were analyzed on degrees of evidence acceptability per HTA body and alignment on evidentiary requirements among HTA bodies. Results Subdomains demonstrating noteworthy differences concerned the acceptability of extrapolation to other populations, class effects, progression-free survival and (other) surrogate endpoints as outcomes, the absence of quality-of-life data, single-arm trials, cross-over trial designs, short trial duration, and the clinical relevance of effect size. Conclusion Alignment can be enhanced to reduce time to decision-making and to improve equity in patient access. Proposed recommendations to achieve this included joint early dialogues, intensified collaboration and exchange between countries, joint relative effectiveness assessments, and the use of access agreements.

Stool DNA testing for early detection of colorectal cancer: systematic review using the HTA Core Model® for Rapid Relative Effectiveness Assessment.

Stool DNA testing for early detection of colorectal cancer (CRC) is a non-invasive technology with the potential to supplement established CRC screening tests. The aim of this health technology assessment was to evaluate effectiveness and safety of currently CE-marked stool DNA tests, compared to other CRC tests in CRC screening strategies in an asymptomatic screening population. The assessment was carried out following the guidelines of the European Network for Health Technology Assessment (EUnetHTA). This included a systematic literature search in MED-LINE, Cochrane and EMBASE in 2018. Manufacturers were asked to provide additional data. Five patient interviews helped assessing potential ethical or social aspects and patients' experiences and preferences. We assessed the risk of bias using QUADAS-2, and the quality of the body of evidence using GRADE. We identified three test accuracy studies, two of which investigated a multitarget stool DNA test (Cologuard®, compared fecal immunochemical test (FIT)) and one a combined DNA stool assay (ColoAlert®, compared to guaiac-based fecal occult blood test (gFOBT), Pyruvate Kinase Isoenzyme Type M2 (M2-PK) and combined gFOBT/M2-PK). We found five published surveys on patient satisfaction. No primary study investigating screening effects on CRC incidence or on overall mortality was found. Both stool DNA tests showed in direct comparison higher sensitivity for the detection of CRC and (advanced) adenoma compared to FIT, or gFOBT, respectively, but had lower specificity. However, these comparative results may depend on the exact type of FIT used. The reported test failure rates were higher for stool DNA testing than for FIT. The certainty of evidence was moderate to high for Cologuard® studies, and low to very low for the ColoAlert® study which refers to a former version of the product and yielded no direct evidence on the test accuracy for ad-vanced versus non-advanced adenoma. ColoAlert® is the only stool DNA test currently sold in Europe and is available at a lower price than Cologuard®, but reliable evidence is lacking. A screening study including the current product version of ColoAlert® and suitable comparators would, therefore, help evaluate the effectiveness of this screening option in a European context.

Stakeholder perspectives on cooperation in the clinical and nonclinical health technology assessment domains.

The aim of this study was to deliver insights from multiple stakeholders into actual and future collaboration for health technology assessment (HTA) in general and in oncology in particular. Eighteen semi-structured interviews were conducted with experts from European HTA bodies (HTAbs), former board members of the European Network for Health Technology Assessment (EUnetHTA), and representatives from the pharmaceutical industry, a regulatory agency, academia, and patient organizations. The stakeholders were asked about their support of the EUnetHTA's intent, about the general strengths and challenges of the EUnetHTA and its Joint Action 3 (JA 3), the strengths and challenges of the clinically oriented HTA collaboration in oncology during JA 3 across the technology life cycle, about future challenges to HTA in oncology with consequences for collaboration, and about collaboration in the economic domains of HTA. The transcribed interviews were analyzed qualitatively. The participants perceived the intention and work quality of the EUnetHTA as positive. The experts described methodological, procedural, and capacity challenges in early dialogues (EDs) and rapid relative effectiveness assessments (REAs) meant to analyze clinical effectiveness in oncology. The majority attached increasing importance to collaboration in the future to cope with the uncertainty of HTA. Several stakeholders also proposed the incorporation of joint postlaunch evidence generation (PLEG) activities. Some gave sporadic suggestions for voluntary nonclinical collaboration as well. Stakeholders' continued readiness to discuss the remaining challenges to and sufficient resources for implementing HTA regulation, as well as further cooperative expansion along the technology life cycle, are necessary for improved HTA collaboration in Europe.

PP70 Enhancing HTA Processes In The Maltese System For Introducing New Medicines

IntroductionMost European countries use Health Technology Assessment (HTA) as input for decisions on reimbursement of (new) medicines. In 2018, following the approval of European Social Funding, the Directorate for Pharmaceutical Affairs (DPA) within the Malta Ministry for Health led a specific Work Package aimed at enhancing knowledge, skills and HTA processes to inform reimbursement decisions.MethodsSince the start of the project, the Institute for Medical Technology Assessment (iMTA) and the National Health Care Institute of the Netherlands (ZIN) collaborate in providing guidance and training to DPA on relative effectiveness assessments (REAs) and pharmacoeconomic assessments (PEAs) of pharmaceuticals. Several activities were organized: site visits, face-to face interviews with stakeholders, a qualitative assessment of the core process of the Maltese system, health economics training and tutorials, meetings introducing the European Network for Health Technology Assessment (EUnetHTA) and Dutch HTA processes, development of a new framework for assessment, and a shadow assessment.ResultsOur assessment identified important methodological challenges and crucial processes interdependencies to optimize within the Maltese system. Based on the learnings, DPA created a template based on the EUnetHTA REA assessment format. IMTA created a template to perform PEAs. Currently, a shadow assessment is ongoing in which DPA performs the REA and PEA of a pharmaceutical using the new templates. ZIN and iMTA will provide feedback to DPA on several aspects, including, but not limited to, the PICO, assessment of therapeutic efficacy and safety, identifying uncertainties, input costs and effects, and formulating a clear discussion of the assessment. If necessary, the templates will be adapted during the process. The project will be evaluated and finalized in 2022.ConclusionsWith the guidance and training provided by iMTA and ZIN, DPA structured and aligned their REA and PEA to enhance their assessment process and to improve the presentation of their HTA report to the two appraisal committees. This project emphasizes the importance of international collaboration to enhance HTA processes within the

PP146 The Use Of Indirect Comparisons For Reimbursement Decision Making In The Netherlands And England

IntroductionReimbursement decision making is based on a relative effectiveness assessment (REA), which may be combined with a cost-effectiveness assessment, by national Health Technology Assessment (HTA) agencies. These assessments are based on clinical data where new interventions are compared to the current standard of care, which may differ between countries. Since most pivotal trials only include a limited number of interventions, indirect treatment comparisons (ITCs) can be used to compare multiple interventions. The aim of this study was to evaluate the use of ITCs in HTA decision making in the Netherlands and England.MethodsAll pharmaceutical assessments published between 2015 and 2019 by the National Health Care Institute (ZIN) and the National Institute for Health and Care Excellence (NICE) were reviewed to determine whether an ITC had been used. For detailed analysis we included all assessments of ZIN using an ITC, and a random sample of assessments of NICE using an ITC (10 assessments per publication year).ResultsBetween 2015 and 2019 a total of 106 and 265 assessments were conducted by ZIN and NICE, respectively. Of these assessments 48 from ZIN and 150 from NICE included an ITC. The detailed analysis showed that pharmaceutical assessments including indirect comparative evidence led to the REA conclusion of similar therapeutic evidence in 57 percent of 48 assessments by ZIN and in 52 percent of 50 assessments by NICE. Reimbursement recommendations including indirect comparative evidence most often resulted in positive recommendations by ZIN (57% assessments), and in restricted recommendations by NICE (50% assessments). Different methods were employed to incorporate indirect comparative evidence, such as naïve ITCs and network meta-analysis.ConclusionsOur results showed a significant variability in the use of ITCs between NICE and ZIN, which may contribute to differences in their recommendations. Further analysis will provide deeper insight in these differences and may provide suggestions for a clearer international guidance on the use of ITCs for HTA.

PP115 Analysis Of Previous Joint Clinical Assessment And Potential Transferability To Four European Countries: Case Study And Conceptual Approach

IntroductionThe European Network for Health Technology Assessment (EUnetHTA) has led together, with member states, several joint actions, including Joint Clinical Assessments (JCA), under the form of project-based voluntary cooperation, which outputs and transferability of those projects in other European countries remains somehow limited. In June 2021, the European Council has reached an agreement on the European Health Technology Assessment (HTA) regulation, which is entering into force gradually. Initially limited to oncology products, then extended to orphan/advanced therapy, and after a five to eight-year additional period, it will apply to all centrally approved products. The JCA will consist of a focused scientific analysis on relative effectiveness assessment, including the health condition, technology description, clinical effectiveness, and safety. These analyses will also include information relating to the degree of certainty. We consider that the evidence appraisal might have limitations, and transferability would not be generalizable. We aim to determine the potential drivers and barriers for HTA transferability in EU4, employing the analysis of a case example where JCA was conducted.MethodsEmploying an oncology JCA, we will compare an HTA analysis conducted in EU4 countries (Germany, France, Italy, and Spain). Overview and background information on countries involved in the JCA, and EU4 HTA system will be provided, followed by HTA outcome and main evidence requirements, reimbursement outcome, and pricing agreements.ResultsStudy results supporting HTA outcomes may focus on the population assessed, the comparator considered, and uncertainty management. A conceptual adaptation about the scope of the EU JCA regulation will be discussed, to understand its potential advantages to individual HTAs in Europe and remaining gaps to effectively inform HTA or decision-making process.ConclusionsThe analysis of pricing and reimbursement outcomes can further help understand potential barriers and drivers for JCA transferability and potential areas of evidence generation requirements.

OP13 EUnetHTA Relative Effectiveness Assessments Of Pharmaceutical And Other Technologies: Procedural Changes Implemented During Joint Action 3

IntroductionThe European Network for Health Technology Assessment (EUnetHTA) Joint Action 3 (JA3) aimed to develop a sustainable European model for scientific and technical collaboration on HTA. It succeeded EUnetHTA JA2, which focused on strengthening practical applications of approaches and tools in the European HTA collaboration. Compared to JA2, several changes in procedures and processes were undertaken throughout JA3 in order to improve the different steps in joint HTA production.MethodsFindings and identified challenges regarding the assessment production processes from JA2 were considered as a basis. In JA3, vast majority of structured and informal feedback was gathered from the assessment teams and project managers via feedback surveys and meetings. Only limited informal feedback from stakeholders (such as patients, health care professionals, and health technology developers) that were involved in EUnetHTA assessments was collected. To this end, experiences were documented and recommendations for a future production process were developed.ResultsDuring the course of JA3, the joint production resulted in 16 pharmaceutical assessments and 27 assessments of other technologies. The latter included medical devices, diagnostics, interventions, and screening. Due to the different context of pharmaceuticals and other technologies, some technology-specific changes needed to be made in their production process. However, the majority of implemented changes were made for both types of technologies to ensure maximum possible alignment in processes. The implemented changes affected several steps in the production process as well as the involvement of stakeholders in EUnetHTA assessments. The production and related project management of assessments was fine-tuned and resulted in clearer, standardized, and comprehensible processes that facilitated transparency and inclusiveness.ConclusionsThe procedural changes led to further standardization and elaboration of assessment production processes in preparation for a future European HTA system under the EU HTA Regulation. However, some methodological challenges remained to be tackled further in the currently ongoing EUnetHTA 21 service contract.

5.M. Workshop: The need to account for vaccine-specificities in the implementation of the EU Regulation on HTA

Abstract In January 2022, the EU Regulation on Health Technology Assessment (HTAR) entered into force. The HTAR provides for the centralized conduct of EU-wide joint clinical assessment (JCA) for oncology treatments (2025 onwards), and for other medicinal products incl. vaccines (2030 onwards). HTA is a multidisciplinary process aiming to disentangle the value of a health technology in comparison to other. It has been used, to different extent across EU countries, to inform vaccine price and reimbursement decisions. Nevertheless, existing assessment frameworks utilized by HTA bodies, incl. National Immunization Technical Advisory Groups (NITAGs), as well as the relative effectiveness assessment (REA) framework developed by the European Network for Health Technology Assessment (EUnetHTA), do not fully account for the specificities of vaccines. In fact, to account for the broad value of vaccines several aspects ought to be considered, including societal benefits. Attention has been paid to these topics for years by the public health community, and in particular EUPHA-HTA, and also Vaccines Europe, a specialized vaccines group within the European Federation of Pharmaceutical Industries and Associations (EFPIA), has recently released a position statement on the need to tackle vaccine specificities in future JCA. Furthermore, as the COVID-19 pandemic has highlighted, such aspects are critical for joint procurement efforts to combat the current and future pandemics. The objective of this workshop is to address the current and future role of HTA in vaccine regulation across EU countries, inform on key specificities of vaccines in respect to JCA and HTA as a whole and discuss opportunities to develop vaccine-specific methodological guidelines within the HTAR. Following an introduction, the following topics will be addressed: • Competent bodies and HTA impact on vaccine decision-making process, incl. a discussion of implementation challenges and potential impacts of the HTAR. • JCA specificities in respect to vaccines and the importance of addressing the whole value of vaccines and vaccinations. • The proposal of Vaccines Europe for guiding principles to account for vaccines specificities in JCA built on the outcomes of an ‘across-the-industry’ project incl. also non-industry experts. The presentations will be followed by a panel discussion to further explore the themes, consider next steps and address questions from the audience. Key messages • The new EU HTAR is paving the way for JCAs of several medicinal products, incl. vaccines necessitating expert guidance in terms of determining their whole value. • Vaccines specificities should be duly considered in JCAs to inform national and EU-wide decision-making for vaccination programmes, reimbursement, and pricing.

Implementation of Health Technology Assessment and Management Course in Undergraduate Program in Biomedical Engineering ITB.

The Undergraduate Program in Biomedical Engineering ITB, Indonesia, introduce the Health Technology Assessment and Management as an elective course in 2021. This course is implemented to support the World Health Assembly that urges the member states to establish national strategies in health technology assessment and management, particularly medical devices. Furthermore, it is designed to give biomedical engineering students a broader insight into their career opportunities. Therefore, this course is delivered by the practitioner and guided by the main lecturer. The course syllabus is developed from the WHO Medical Devices Technical Series and European Network for Health Technology Assessment. It tries to implement HTA Core Model for Rapid Relative Effectiveness Assessments. A questionnaire is used to measure the students' perception of the course implementation. Moreover, it is used to obtain the students' comments and feedback. The course that is delivered by the practitioner not only gives the course content but also the context. After attending the course, students have a broader insight into the career opportunities as biomedical engineers in Indonesia.

Differences in Evidentiary Requirements Between European Medicines Agency and European Health Technology Assessment of Oncology Drugs—Can Alignment Be Enhanced?

ObjectivesNational health technology assessments (HTAs) across Europe show differences in evidentiary requirements from assessments by the European Medicines Agency (EMA), affecting time to patient access for drugs after marketing authorization. This article analyzes the differences between EMA and HTA bodies’ evidentiary requirements for oncology drugs and provides recommendations on potential further alignment to minimize and optimally manage the remaining differences. MethodsInterviews were performed with representatives and drug assessment experts from EMA and HTA bodies to identify evidentiary requirements for several subdomains and collect recommendations for potentially more efficiently addressing differences. A comparative analysis of acceptability of the evidence by EMA and the HTA bodies and for potential further alignment between both authorities was conducted. ResultsAcceptability of available evidence was higher for EMA than HTA bodies. HTA bodies and EMA were aligned on evidentiary requirements in most cases. The subdomains showing notable differences concerned the acceptance of limitation of the target population and extrapolation of target populations, progression-free survival and (other) surrogate endpoints as outcomes, cross-over designs, short trial duration, and clinical relevance of the effect size. Recommendations for reducing or optimally managing differences included joint early dialogues, joint relative effectiveness assessments, and the use of managed entry agreements. ConclusionsDifferences between assessments of EMA and HTA bodies were identified in important areas of evidentiary requirements. Increased alignment between EMA and HTA bodies is suggested and recommendations for realization are discussed.

Involvement of Patients and Medical Professionals in the Assessment of Relative Effectiveness: A Need for Closer Cooperation

ObjectivesInvolvement of patients and medical professionals in assessment of relative effectiveness (relative effectiveness assessment) contributes to an efficient and effective health technology assessment (HTA) process and supports acceptance and implementation of the outcome. This study aimed to analyze stakeholder involvement in assessing relative effectiveness and how the parties involved value this collaboration. MethodsThis is a document analysis of all drug assessments completed in 2019 (20) by the public HTA agency of The Netherlands, enriched with semistructured interviews with employees of the HTA agency (18) and representatives of patient (5) and medical (11) associations involved in these assessments. Data were analyzed, coded, and categorized. ResultsIn almost half of the assessments, there was no coordination with the medical associations at the start of the relative effectiveness assessment and no patient associations involved in this phase. During the assessment procedure, patient and medical associations were always asked to comment on the draft report. Nevertheless, the strict 5-day deadline that the HTA agency uses as a response period often hampered a proper response and involvement. According to interviewees of the HTA agency, this leads to a great diversity in the substantive quality of their input. Patient and medical associations indicated that the HTA agency relies too much on “paper knowledge,” which leads to a (perceived) lack of alignment with clinical practice. ConclusionsThe limited involvement results in a lack of coordination and mutual trust. Optimizing involvement of patients and medical professionals in HTA practice requires effort from all parties involved. Procedural adjustments and better coordination, especially at the start of the assessment, would probably improve cooperation.

Strengthening the Interface of Evidence-Based Decision Making Across European Regulators and Health Technology Assessment Bodies

ObjectivesAccess to medicines in Europe depends on a benefit-risk decision taken by regulators and a relative effectiveness assessment performed by health technology assessment bodies (HTABs) to inform, as one element, a reimbursement decision. Although various similarities in evidence needs exist, understanding of their needs is currently suboptimal and therefore the evidence generated does not always meet their needs. Subsequently, delays in decision making can be expected, negatively affecting access. To overcome this, this study reviewed the evidentiary needs of European regulators and HTABs at European level and analyzed how their collaboration can further facilitate optimal evidence generation plans, evidence use, and evidence presentation. MethodsThrough systematic literature review, expert interviews, and pairwise comparison of assessment reports by the European Medicines Agency and European network for health technology assessment, respective clinical evidence requirements and impact of product-specific collaboration between European Medicines Agency and HTABs were established. ResultsClinical evidence needs are quite similar but differences exist in comparator choice, preferred efficacy endpoints, and target population. Results of the impact of collaboration to date were mixed: preapproval joint advice procedures were successful and highly valued by all stakeholders; information exchange at the time of regulatory decision is coming together, yet the European Public Assessment Report can be further optimized; and collaboration on postlicensing evidence generation requirements shows potential but needs solidifying. ConclusionsThese findings demonstrate the potential to further improve the evidence utilization across stakeholders to avoid duplication and streamline decision making, to ultimately improve access to medicines for European patients.

EUnetHTA relative effectiveness assessments: efforts to increase usability, transparency and inclusiveness.

The objective of the European Network for Health Technology Assessment (EUnetHTA) Joint Action 3 (JA3) was to develop a sustainable European model for future collaboration on HTA, by reducing duplication in HTA production and increasing patient access to health technologies. Compared to the previous JA2, several procedural changes were made aiming to increase usability, transparency, and inclusiveness of relative effectiveness assessments (REAs). This article presents and highlights these changes, explains their rationale as well as their influence on HTA production. Feedback from REA teams and project managers was collected. At the end of JA3, all lessons learned were mapped, resulting in a set of recommendations for a future REA production process. In JA3, forty-three EUnetHTA REAs have been produced. Efforts to increase the usability of the REAs were made by focussing on the needs of REA producers and users (HTA agencies) and by increasing stakeholder involvement. Huge steps were taken with regard to transparency, which was achieved through publication of guidances, templates, and up-to-date information on the EUnetHTA website. In an attempt to improve inclusiveness, (stakeholder) interaction and involvement as well as feedback procedures were enhanced and streamlined. The fine-tuned project management brought all aspects together and facilitated a consistent and reliable workflow. Despite that HTA agencies have different national requirements, the procedural changes made in JA3 proved to counteract some of these challenges. Nevertheless, it is of utmost importance that further perceived methodological differences are being resolved to ensure a strong base for future European collaboration on REA production.

Addressing uncertainty in relative effectiveness assessments by HTA organizations.

This study outlines the ways in which different health technology assessment (HTA) organizations deal with uncertainty in relative effectiveness assessments (REAs), using the GRADE framework as a common reference. Guidelines regarding REA and uncertainty assessment methods and three most recent HTA reports (as of April 2020) of seven HTA organizations in Germany, England and Wales, France, the Netherlands, Europe (EUnetHTA), the USA, and Canada were included. First, it was analyzed how each organization addressed uncertainty on the following levels of evidence: (i) individual studies, (ii) body of evidence for one outcome, (iii) body of evidence across all outcomes, and (iv) added net benefit. Second, the extent to which HTA organizations considered the eight domains of certainty of evidence defined by GRADE was assessed. For individual studies, checklists were the most common approach to express uncertainty (4/7 organizations). Uncertainty in the body of evidence for all outcomes and in added benefit was combined in a single conclusion by five organizations. All organizations reported on at least 4/5 downgrading domains of GRADE, while the three upgrading domains were reported less. The operationalization of the assessment of multiple domains was unclear due to vague or absent guidelines. HTA organizations consider most domains of the GRADE framework, but approaches to assess uncertainty within REAs on different levels of evidence differ substantially between organizations. More alignment and guidance on the best methods to deal with uncertainty within HTA could lead to more clarity for stakeholders and to more aligned reimbursement recommendations.

Transferability of a EUnetHTA relative effectiveness assessment to low- and middle-income countries setting.

In 2020, European Network for Health Technology Assessment (EUnetHTA) published a relative effectiveness analysis (REA) of Pretomanid in combination with Bedaquiline and Linezolid for the treatment of extensively drug-resistant (XDR) or treatment-intolerant or nonresponsive multidrug-resistant (MDR) tuberculosis (TB) (REA PTJA14). This REA may have a significant value for low- and middle-income countries (LMICs) outside Europe, particularly those with a high burden of drug-resistant TB. This commentary focuses on whether the REA PTJA14 can be transferred and to what extent a REA can be translated to LMICs context outside Europe. We found that the assessments on the clinical effectiveness and risks of bias reported in REA PTJA14 are useful for LMICs outside Europe. The highly standardized management of TB will support the applicability of the REA to LMICs outside of Europe. Transferring this REA can reduce workload and efficiently use limited resources to conduct health technology assessment (HTA). However, the transfer should consider several critical issues, including variations in health system delivery and clinical practice and setting-specific constraints. In the TB context, the differences in the current standard treatment for XDR or nonresponsive MDR TB, resources availability for drug-resistant TB management, and how healthcare is delivered in the countries can complicate the applicability of the REA PTJA14. Given that LMICs have limitations in doing HTA, it is now critical to develop standard guidelines for transferring REA or other HTA results from high-income countries or other LMICs to maximize the benefits of the REA for LMICs outside Europe.

Reported Challenges in Health Technology Assessment of Complex Health Technologies

ObjectivesWith complex health technologies entering the market, methods for health technology assessment (HTA) may require changes. This study aimed to identify challenges in HTA of complex health technologies. MethodsA survey was sent to European HTA organizations participating in European Network for HTA (EUnetHTA). The survey contained open questions and used predefined potentially complex health technologies and 7 case studies to identify types of complex health technologies and challenges faced during HTA. The survey was validated, tested for reliability by an expert panel, and pilot tested before dissemination. ResultsA total of 22 HTA organizations completed the survey (67%). Advanced therapeutic medicinal products (ATMPs) and histology-independent therapies were considered most challenging based on the predefined complex health technologies and case studies. For the case studies, more than half of the reported challenges were “methodological,” equal in relative effectiveness assessments as in cost-effectiveness assessments. Through the open questions, we found that most of these challenges actually rooted in data unavailability. Data were reported as “absent,” “insufficient,” “immature,” or “low quality” by 18 of 20 organizations (90%), in particular data on quality of life. Policy and organizational challenges and challenges because of societal or political pressure were reported by 8 (40%) and 4 organizations (20%), respectively. Modeling issues were reported least often (n = 2, 4%). ConclusionsMost challenges in HTA of complex health technologies root in data insufficiencies rather than in the complexity of health technologies itself. As the number of complex technologies grows, the urgency for new methods and policies to guide HTA decision making increases.

A Review of the Evidence on Attitudes, Perceived Impacts and Motivational Factors for European Member State Collaboration for Pricing and Reimbursement of Medicines: Time for the EEA Member States to Apply Their Experience and Expertise in Evidence-Based Decision Making to Their Current Pharmaceutical Policy Challenges.

In 2018/2019 there were a number of initiatives for collaboration between Member States in the European Economic Area (EEA) and the European Commission published a Proposal for a Regulation on Health Technology Assessment. In view of the perceived benefits from collaboration, the experiences and challenges of these collaborative initiatives and the possible implications of the proposed legislation, a study of the evidence on attitudes, perceived impacts and the motivational factors towards European Member State collaboration regarding the pricing and reimbursement of medicines was conducted. This study adopted an evidence–based management approach by Barends and Rousseau. The main findings showed that Member States differed in their motivation for collaboration for different pharmaceutical activities. Member States favoured voluntary co-operation for all activities of pricing and reimbursement except for relative effectiveness assessments where Member State authorities had divergent attitudes and prioritised activities related to the sustainability of their healthcare systems and access to medicines. Contrastingly pharmaceutical companies strongly favoured mandatory cooperation for evaluation. Member States motivation for collaboration was highly dependent on the purpose, political will, implementation climate and cultural factors. Currently, with the experiences of ongoing collaborations, following the progress of the discussion at Council, and with a number of inititatives for new pharmaceutical strategy and policy, it is proposed that Member States use their trust, expertise and knowledge of application of evidence-based decision making for pricing and reimbursement of medicines and apply it to decide the future model for Member State collaboration. The applicability of principles of evidence-based management to pharmaceutical policy can be used as a starting point.