European Health Technology Assessment Research Articles

The development of a roadmap for Health Technology Assessment implementation in Moldova

Abstract* Background Moldova strives for universal health coverage (UHC) and considers health technology assessment (HTA) an important policy instrument to inform the choice of services that should be covered to progressively realize UHC. It plays a key role in determining which technologies are coved, considering various dimensions often including cost-effectiveness, budget impact, and feasibility. This paper reports on work undertaken to develop a roadmap for HTA implementation, using evidence-informed deliberative processes (EDPs) as the guiding framework. Methods Between 2020 and 2022, we undertook several activities that informed the roadmap. We conducted a needs assessment and an assessment of European HTA best practices using a combination of desk research, interviews and surveys. We then conducted a document review of six selected HTA systems, complimented by expert interviews from three neighbouring countries. Results The roadmap provided a suggested HTA process, which included instructions on how to execute the different steps of the EDP framework to enhance the legitimacy of HTA-informed decision-making. The roadmap encompasses several aspects such as how to organize stakeholder involvement, how to identify and operationalize decision criteria, and how to make the decision process transparent. Guidance was given to the country on establishing a legal framework for HTA; a communication strategy; strengthening capacity and leveraging HTA expertise. The country was also given tailored advice on the positioning of the HTA-agency, first as an entity within the Ministry of Health or the National Agency of Public Health but with the long-term vision for it to be independent of any state institution. Conclusions Despite several challenges, including a partial disruption because of the Covid-19 pandemic, the development of the roadmap for HTA implementation was completed and agreed upon by the Ministry of Health in Moldova. This work has helped motivate and support the country in HTA institutionalization.

An Inclusive Civil Society Dialogue for Successful Implementation of the EU HTA Regulation: Call to Action to Ensure Appropriate Involvement of Stakeholders and Collaborators.

Stakeholder involvement has long been considered a success factor for a joint European health technology assessment (HTA) process, and its relevance is now anchored in the EU HTA Regulation's (EU HTAR) legislative wording. Therefore, we aimed to explore the roles, challenges, and most important activities to increase the level of involvement per stakeholder group. At the 2022 Fall Convention of the European Access Academy (EAA), working groups addressed the involvement of patients, clinicians, regulators, health technology developers (HTD), and national HTA bodies and payers within the EU HTA process. Each working group revisited the pre-convention survey results, determined key role characteristics for each stakeholder, and agreed on the most important activities to fulfill the role profile. Finally, the activities suggested per group were prioritized by plenary group. The prioritized actions for patients included training and capacity building, the establishment of a patient involvement committee, and the establishment of a patient unit at the EC secretariat. For clinicians, it included alignment on evidence assessment from a clinical vs. HTA point of view, capacity building, and standardization of processes. The most important actions for regulators are to develop joint regulatory-HTA guidance documents, align processes and interfaces under the regulation, and share discussions on post-licensing evidence generation. HTDs prioritized scientific advice capacity and the review of the scoping process, and further development of the scope of the assessment report fact checks. The top three actions for national HTA bodies and payers included clarification on the early HTD dialogue process, political support and commitment, and clarification on financial support. Addressing the activities identified as the most important for stakeholders/collaborators in the EU HTA process (e.g., in the implementation of the EU HTA Stakeholder Network and of the guidance documents developed by the EUnetHTA 21 consortium) will be key to starting an "inclusive civil society dialogue", as suggested by the European Commission's Pharmaceutical Strategy.

Tools for assessing quality of studies investigating health interventions using real-world data: a literature review and content analysis

ObjectivesWe aimed to identify existing appraisal tools for non-randomised studies of interventions (NRSIs) and to compare the criteria that the tools provide at the quality-item level.DesignLiterature review through three approaches:...

Pediatric Market Access: A Qualitative Study.

This qualitative study aims to analyze current PM regulation and market access requirements and proposes potential solutions to mitigate current challenges. Twenty-two semi-structured interviews were conducted with experts from pharmaceutical industry, regulatory authorities, national health technology assessment (HTA) bodies, pediatricians, and academia from the Netherlands (NL), Germany (DE), the United Kingdom (UK), and France (FR) to get insight into the pediatric research, the regulatory and reimbursement processes, challenges, and solutions. Themes for further testing were developed on how to facilitate pediatric market access. Atlas.ti 9 was used to analyze the findings. Heterogeneity in requirements for the European Medicines Agency (EMA) and HTA approvals are noted. By example, DE grants direct reimbursement after regulatory approval, the other countries require additional reimbursement which generate delays and challenges in patient access after marketing authorization. Key components in facilitating PM market access include multi-stakeholder collaboration, transparency, patient representatives, informed consent guidance, real-world evidence, and appropriate clinical trial designs. Pricing models based on the economic capabilities of individual countries could further reduce delays and challenges in market access. The additional specific pediatric incentives should be taken as best practice to encourage innovation in pediatric conditions. This study highlights differences in requirements for regulatory and reimbursement approval, along with international differences in pricing and reimbursement procedures for pediatric market access.

Opportunities and Challenges for Decentralized Clinical Trial Approaches: European Health Technology Assessment Perspective

ObjectivesDecentralized clinical trial (DCT) approaches are clinical trials in which some or all trial activities take place closer to participants’ proximities instead of a traditional investigative site. Data from DCTs may be used for clinical and economic evaluations by health technology assessment (HTA) bodies to support reimbursement decision making. This study aimed to explore the opportunities and challenges for DCT approaches from an HTA perspective by interviewing representatives from European HTA bodies. MethodsWe conducted semistructured interviews with 25 European HTA representatives between September 2022 and February 2023, and transcripts were analyzed after thematic analysis. ResultsTwo main themes were identified from the data relating to (1) DCT approaches in HTA and (2) trial-level acceptance and relevance. Experience with assessing DCTs was limited and a variety of knowledge about DCTs was observed. The respondents recognized the opportunity of DCTs to reduce recall bias when participant-reported outcome data can be collected more frequently and conveniently from home. Concerns were expressed about the data quality when participants become responsible for data collection. Despite this challenge, the respondents recognized the potential of DCTs to increase the generalizability of results because data can be collected in a setting reflective of the everyday situation potentially from a more diverse participant group. ConclusionsDCTs could generate relevant results for HTA decision making when data are collected in a real-world setting from a diverse participant group. Increased awareness of the opportunities and challenges could help HTA assessors in their appraisal of DCT approaches.

PP52 Will Joint European Health Technology Assessment Provide Additional Benefits Over Individual Country-wise Assessments?

IntroductionIn December 2021 the European Union (EU) Health Technology Assessment (HTA) Regulation, a key pillar of the EU Pharmaceutical Strategy, was adopted by the Council and the European Parliament. The focus areas of Joint HTA Cooperation include Joint Clinical Assessments (JCA), Joint Scientific Consultations (JSC), and joint early-stage horizon scanning. The European HTA regulation will be adopted in a stepwise approach and from 2030 onwards, all products (drugs, high-risk medical devices, and in vitro diagnostics) approved in all indications will be subjected to JCA in EU.MethodsA targeted literature research was performed for policies and the European Network for HTA (EUnetHTA) methodological guidelines describing the HTA methods including scoping process, comparators, endpoints, the applicability of evidence, and validity of clinical studies. Additionally, the anticipated opportunities and challenges were also summarized with respect to these methods.ResultsEUnetHTA put forward a timeline for different activities over the next three years as part of the new EU HTA Regulation, including key deadlines for ongoing EUnetHTA consultations on the processes and methods. EUnetHTA will set up a new ecosystem across the EU as it aims to reduce duplication and time to access by supplementing multiple national clinical assessments with a joint central assessment. In any case, assessment of added value and pricing and reimbursement decisions will still occur at the national level.Additionally, EU HTA may promote harmonization of processes, standards, and evidence requirements, which will increase predictability and simplify evidence requirements. However, differences in clinical practice, standard of care, and national priorities may lead to assessments that are not generalizable to all Member States.ConclusionsThe joint EU HTA cooperation will benefit countries which have less developed or do not have established HTA expertise or infrastructure. However, the JCA process could result in increased requirements for clinical evidence generation as relative effectiveness and relevance of outcomes to patients gain further importance for products to successfully gain access across countries.

PP97 How Health Technology Agencies Estimate Target Population Size For Medical Devices: The Example Of Spinal Cord Stimulation

IntroductionSpinal cord stimulation (SCS) is an effective and safe option for patients with refractory neuropathic pain, with positive health technology assessment (HTA) recommendations across Western Europe, yet SCS uptake remains low. Estimating target patient populations within HTAs may impact upon medical device uptake, pricing and access. However, there is a dearth of information on how this is typically conducted. This study aimed to compare the approaches of Western Europe HTA agencies for estimating the target population for SCS.MethodsA survey was conducted among members of the Health Economics and Reimbursement function from Western Europe to collect country-specific information on how HTA agencies assess the target population of medical devices (MD). The estimations of the target population for SCS were extracted from HTA publications and compared.ResultsEight Health Economics and Reimbursement (HER) Analysts from France, Germany, the United Kingdom (UK), Belgium, Spain, Italy, Sweden and Norway completed the survey. HTA Agencies in France, UK and Belgium routinely ask for epidemiological data in the manufacturer submission, whereas in Germany, Sweden and Norway the request is dependent on the type of HTA submission. All HTA agencies, except NICE (UK), perform an independent estimation of the target population. HTA agencies in Germany and UK typically use epidemiological data from industry. In all countries, the estimation of the target population may indirectly impact the price of the MD, especially when budget impact analysis indicates a potential high use of healthcare resources. Only France, Belgium and UK have published HTA recommendations about SCS, however the estimated target population, nor the number of patients with refractory neuropathic pain, is not always included. Only the French and UK HTA agencies publish an approximation of the target population for SCS using the number of patients implanted every year.ConclusionsThis study showed there is a lack of harmonization between Western European HTA agencies’ guidelines on the inclusion and estimation of target patient populations. The new EU HTA Regulation should help to address this situation.

SA79 Opportunities and Challenges for Decentralized Clinical Trials: European Health Technology Assessment Perspective

SA79 Opportunities and Challenges for Decentralized Clinical Trials: European Health Technology Assessment Perspective

Differences in evidentiary requirements for oncology drug effectiveness assessments among six European health technology assessment bodies — can alignment be improved?

ABSTRACT Objectives Evidentiary requirements for relative effectiveness assessment vary among European health technology assessment (HTA) bodies, affecting the time to HTA decision-making and potentially delaying time to patient access. Improved alignment may reduce this time; therefore, we aim to analyze the differences in evidentiary requirements for oncology drug assessments among European HTA bodies and provide recommendations toward an increased alignment. Methods Interviews were conducted with stakeholders in drug assessments of Italy, the Netherlands, Poland, Portugal, England and Wales, and Sweden about evidentiary requirements for several subdomains to identify differences and obtain recommendations for addressing differences. The interview results were analyzed on degrees of evidence acceptability per HTA body and alignment on evidentiary requirements among HTA bodies. Results Subdomains demonstrating noteworthy differences concerned the acceptability of extrapolation to other populations, class effects, progression-free survival and (other) surrogate endpoints as outcomes, the absence of quality-of-life data, single-arm trials, cross-over trial designs, short trial duration, and the clinical relevance of effect size. Conclusion Alignment can be enhanced to reduce time to decision-making and to improve equity in patient access. Proposed recommendations to achieve this included joint early dialogues, intensified collaboration and exchange between countries, joint relative effectiveness assessments, and the use of access agreements.

Perspectives on how to build bridges between regulation, health technology assessment and clinical guideline development: a qualitative focus group study with European experts

ObjectiveImproving synergy among regulation, health technology assessment (HTA) and clinical guideline development is relevant as these independent processes are building on shared evidence-based grounds. The two objectives were first to...

HPR44 Evolving PICO Characteristics in the Early Oncology Treatment Lifecycle: Differences from Trials to Treatment Guidelines

Under the new European health technology assessment (HTA) legislation, European countries will largely need to agree on the PICO (population, intervention, comparator, outcome) elements in HTA assessments. Additionally, cross-stakeholder alignment grows in importance due to our ever-evolving healthcare landscape, such as smaller populations and increasingly advanced products. The PICO therefore becomes more subject to variation among decision-makers. This study assessed how the PICO elements evolve over the early treatment lifecycle of oncology drugs and align between stakeholders on a global level.

Challenges and Solutions for the Benefit Assessment of Tumor-Agnostic Therapies in Germany

ObjectivesPrecision medicine is increasingly important in cancer treatment. Tumor-agnostic therapies are used regardless of tumor entity because they target specific biomarkers in tumors. In Germany, the benefit assessment of oncological pharmaceuticals has traditionally been entity specific. Thus, the assessment of tumor-agnostic therapies leaves stakeholders with various challenges. Our aim was to systematically identify challenges and possible solutions for the benefit assessment of therapies in tumor-agnostic indications using a 2-step sequential qualitative approach. MethodsTo identify relevant challenges, we conducted qualitative interviews with different stakeholders who were involved in previous benefit assessments of tumor-agnostic therapies in Germany. To identify possible solutions for these challenges, we systematically searched MEDLINE, Embase, and the websites of European health technology assessment bodies for relevant literature. ResultsWe identified 9 categories of challenges of which the following were deemed particularly relevant: the absence of direct comparative studies, challenges regarding the use of basket studies and indirect comparisons, challenges in determining the appropriate comparative therapy in a tumor-agnostic indication, and challenges on the system side. Seven categories of solutions were identified, including an increased use of real-world evidence, making conditional decisions in the context of systematic reassessments, splitting the field of application, and finding (new) ways to design and analyze basket studies. ConclusionA range of possible solutions, which can help to meet the identified challenges in Germany, have been found. Future research should investigate the acceptance and feasibility of these solutions.

Stakeholder perspectives on cooperation in the clinical and nonclinical health technology assessment domains.

The aim of this study was to deliver insights from multiple stakeholders into actual and future collaboration for health technology assessment (HTA) in general and in oncology in particular. Eighteen semi-structured interviews were conducted with experts from European HTA bodies (HTAbs), former board members of the European Network for Health Technology Assessment (EUnetHTA), and representatives from the pharmaceutical industry, a regulatory agency, academia, and patient organizations. The stakeholders were asked about their support of the EUnetHTA's intent, about the general strengths and challenges of the EUnetHTA and its Joint Action 3 (JA 3), the strengths and challenges of the clinically oriented HTA collaboration in oncology during JA 3 across the technology life cycle, about future challenges to HTA in oncology with consequences for collaboration, and about collaboration in the economic domains of HTA. The transcribed interviews were analyzed qualitatively. The participants perceived the intention and work quality of the EUnetHTA as positive. The experts described methodological, procedural, and capacity challenges in early dialogues (EDs) and rapid relative effectiveness assessments (REAs) meant to analyze clinical effectiveness in oncology. The majority attached increasing importance to collaboration in the future to cope with the uncertainty of HTA. Several stakeholders also proposed the incorporation of joint postlaunch evidence generation (PLEG) activities. Some gave sporadic suggestions for voluntary nonclinical collaboration as well. Stakeholders' continued readiness to discuss the remaining challenges to and sufficient resources for implementing HTA regulation, as well as further cooperative expansion along the technology life cycle, are necessary for improved HTA collaboration in Europe.

PP115 Analysis Of Previous Joint Clinical Assessment And Potential Transferability To Four European Countries: Case Study And Conceptual Approach

IntroductionThe European Network for Health Technology Assessment (EUnetHTA) has led together, with member states, several joint actions, including Joint Clinical Assessments (JCA), under the form of project-based voluntary cooperation, which outputs and transferability of those projects in other European countries remains somehow limited. In June 2021, the European Council has reached an agreement on the European Health Technology Assessment (HTA) regulation, which is entering into force gradually. Initially limited to oncology products, then extended to orphan/advanced therapy, and after a five to eight-year additional period, it will apply to all centrally approved products. The JCA will consist of a focused scientific analysis on relative effectiveness assessment, including the health condition, technology description, clinical effectiveness, and safety. These analyses will also include information relating to the degree of certainty. We consider that the evidence appraisal might have limitations, and transferability would not be generalizable. We aim to determine the potential drivers and barriers for HTA transferability in EU4, employing the analysis of a case example where JCA was conducted.MethodsEmploying an oncology JCA, we will compare an HTA analysis conducted in EU4 countries (Germany, France, Italy, and Spain). Overview and background information on countries involved in the JCA, and EU4 HTA system will be provided, followed by HTA outcome and main evidence requirements, reimbursement outcome, and pricing agreements.ResultsStudy results supporting HTA outcomes may focus on the population assessed, the comparator considered, and uncertainty management. A conceptual adaptation about the scope of the EU JCA regulation will be discussed, to understand its potential advantages to individual HTAs in Europe and remaining gaps to effectively inform HTA or decision-making process.ConclusionsThe analysis of pricing and reimbursement outcomes can further help understand potential barriers and drivers for JCA transferability and potential areas of evidence generation requirements.

HTA43 Multi-Country HTA Initiatives Versus National HTA Processes: The Optimal Reimbursement Strategy

Health technology assessment (HTA) plays a crucial role in the reimbursement decision-making process in many countries, but recommendations vary widely throughout jurisdictions, even for the same drug. To harmonise methodological standards and foster collaboration among European HTA bodies, several joint HTA initiatives have been established. Our review aimed to provide a concise overview and comparison of joint HTA and drug price collaborations and to gain an understanding of how these impact pricing and market access.

OP12 Post-Launch Evidence Generation Among Health Technology Assessment Bodies In Europe

IntroductionThe need for timely access to innovative technologies has placed a special focus on the development of policies and practices that can guarantee the availability whilst ensuring the safety of these technologies after launch or licensure. The aim of this paper is to present and discuss Post-Launch Evidence Generation (PLEG) practices among health technology assessment (HTA) bodies at the European level to explore cross-border collaboration opportunities.MethodsIn December 2019, a survey composed of nine closed-ended questions with multiple choice answers about the PLEG practices in each country was sent to 25 partners of the European Network of Health Technology Assessment (EUnetHTA) Joint Action 3. In addition to the survey, the national practices were discussed during a face-to-face meeting with all partners of the dedicated work package. A quantitative analysis and a qualitative synthesis of the results was carried out.ResultsTwelve HTA bodies completed the survey. Of these, 11 reported procedures in place for official PLEG requests. In nine of the agencies, the requests are made at the time of the assessment/appraisal. Data collection and analysis mainly lies with companies for pharmaceuticals (60%) while it is more the responsibility of the HTA body for medical devices (75%). Only one agency reported owning the data and being able to exchange the data without asking permission. During the face-to face discussions, it was acknowledged that PLEG practices differ between countries depending on the topic concerned, but most rely on the usage of registries (mainly disease registries) for data collection. Most agencies estimated that a European collaboration could take place.ConclusionsPLEG practices are in the remit of many European HTA bodies. Data sharing should be anticipated as only some own the data and can exchange them without asking permission. European collaboration on PLEG could commence once the evidence gaps have been defined or during the production of the HTA reports in the case of joint assessments.

Shaping a research agenda to ensure a successful European health technology assessment: insights generated during the inaugural convention of the European Access Academy

ObjectivesKey challenges for a joint European Health Technology Assessment (HTA) include consolidated approaches towards the choice of adequate comparator(s), selection of endpoints that are relevant to patients with a given disease, dealing with remaining uncertainties as well as transparent and consistent management of related processes. We aimed to further crystallize related core domains within these four areas that warrant further research and scrutiny.MethodsBuilding on the outcomes of a previously conducted questionnaire survey, four key areas, processes, uncertainty, comparator choice and endpoint selection, were identified. At the inaugural convention of the European Access Academy dedicated working groups were established defining and prioritizing core domains for each of the four areas. The working groups consisted of ~ 10 participants each, representing all relevant stakeholder groups (patients/ clinicians/ regulators/ HTA & payers/ academia/ industry). Story books identifying the work assignments were shared in advance. Two leads and one note taker per working group facilitated the process. All rankings were conducted on an ordinal Likert Response Scale scoring from 1 (low priority) to 7 (high priority).ResultsIdentified key domains include for processes: i) address (resource-) challenge of multiple PICOs (Patient/ Intervention/ Comparator/ Outcomes), ii) time and capacity challenges, iii) integrating all involved stakeholders, iv) conflicts and aligning between different multi-national stakeholders, v) interaction with health technology developer; for uncertainty: i) early and inclusive collaboration, ii) agreement on feasibility of RCT and acceptance of uncertainty, iii) alignment on closing evidence gaps, iv) capacity gaps; for comparator choice: i) criteria for the choice of comparator in an increasingly fragmented treatment landscape, ii) reasonable number of comparators in PICOs, iii) shape Early Advice so that comparator fulfils both regulatory and HTA needs, iv) acceptability of Indirect Treatment Comparisons (ITC), v) ensure broad stakeholder involvement in comparator selection; for endpoint selection: i) approaching new endpoints; ii) patient preferences on endpoints; iii) position of HTA and other stakeholders; iv) long-term generation and secondary use of data; v) endpoint challenges in RCTs.ConclusionsThe implementation of a joint European HTA assessment is a unique opportunity for a stronger European Health Union. We identified 19 domains related to the four key areas, processes, uncertainty, comparator choice and endpoint selection that urgently need to be addressed for this regulation to become a success.

2.G. Workshop: The development of HTA collaboration in Europe: where we are and where we are going

Abstract Health decision-making should be always well-informed. This draws the attention on the need to make the best use of available evidence in respect to different clinical and non-clinical issues to make a value-based allocation of resources. Health Technology Assessment (HTA) is a method for supporting evidence-informed health policymaking as it is defined as a multidisciplinary process that uses explicit methods to determine the value of a health technology at different points in its lifecycle. The role of HTA is of utmost importance at European Union level considering that citizens have the right to access healthcare in any EU country with reimbursement by their home country. Therefore, Member States (MS) cooperation in the evaluation of health technologies is envisaged to ensure safe, high-quality, and efficient healthcare. Several efforts have been put in place at EU level to make this collaboration possible. The cross-border healthcare directive 2011/24/EU have led to the establishment of a voluntary network of MS to facilitate their cooperation and the exchange of scientific information. This network has been supported by another network, namely the European Health Technology Assessment Network (EUnetHTA) that was launched in 2006 with the aim to increase the contribution of HTA to decision-making in EU MS, strengthen the link between HTA and health policymaking, and reduce the overlap and duplication of HTA efforts by promoting a more effective use of resources. EUnetHTA has created a long-lasting Joint Action that has developed principles, methodological guidelines, as well as functional online tools for the sharing production of HTA. In the meantime, starting from 2018, a proposal for an EU regulation on HTA has been discussed within the European Commission first and the European Council and Parliament later. The discussion led to the approval of the Regulation (EU) 2021/2282 on health technology assessment (HTAR) that entered into force on 11 January 2022 and will apply from 12 January 2025 onwards. The HTAR calls for mandatory participation in and use of joint clinical assessments at MS level opening a new era for HTA at EU level. In order to make public health professionals ready for the challenge, this skill-building seminar will provide the audience with basic knowledge in respect to the objective, the application, the methods, and the history of HTA at European level. After two formal presentations on these issues, a case study will lead the following interactive discussion with the audience. Key messages • Health decisions should be informed by the determination of the value of health technologies through shared methods, such as Health Technology Assessment. • Collaboration on Health Technology Assessment at European level has had a long-lasting story and is now looking out into a new challenging era that will be the EU regulation being implemented.

A call to action to harmonize patient-reported outcomes evidence requirements across key European HTA bodies in oncology.

Patient-reported outcome (PRO) data are increasingly being included in Health Technology Assessment (HTA) submissions for oncology drugs. This study aims to provide differences in PRO evidence requirements in oncology across key HTA bodies and calls for its harmonization. Method guidance provided by HTA bodies in Germany, France and the UK, and analysis of HTA reports of 20 oncology case studies were evaluated in this review. Differences exist between HTA bodies regarding guidance on how PRO data should be collected, reportedand analyzed as well as how the data are reviewed and considered in oncology HTAs. HTA bodies can play a key role to harmonize PRO method guidance in collaboration with regulators and sponsors.

Real-world evidence in health technology assessment of high-risk medical devices: Fit for purpose?

Health technology assessment (HTA) of medical devices (MDs) increasingly rely on real‐world evidence (RWE). The aim of this study was to evaluate the type and the quality of the evidence used to assess the (cost‐)effectiveness of high risk MDs (Class III) by HTA agencies in Europe (four European HTA agencies and EUnetHTA), with particular focus on RWE. Data were extracted from HTA reports on the type of evidence demonstrating (cost‐)effectiveness, and the quality of observational studies of comparative effectiveness using the Good Research for Comparative Effectiveness principles. 25 HTA reports were included that incorporated 28 observational studies of comparative effectiveness. Half of the studies (46%) took important confounding and/or effect modifying variables into account in the design and/or analyses. The most common way of including confounders and/or effect modifiers was through multivariable regression analysis. Other methods, such as propensity score matching, were rarely employed. Furthermore, meaningful analyses to test key assumptions were largely omitted. Resulting recommendations from HTA agencies on MDs is therefore (partially) based on evidence which is riddled with uncertainty. Considering the increasing importance of RWE it is important that the quality of observational studies of comparative effectiveness are systematically assessed when used in decision‐making.