Health technology assessments and real-world evidence: tell us what you want, what you really, really want.

Abstract

Journal of Comparative Effectiveness ResearchVol. 11, No. 5 CommentaryOpen AccessHealth technology assessments and real-world evidence: tell us what you want, what you really, really wantFrank Griesinger, Oliver Cox, Cormac Sammon, Sreeram V Ramagopalan & Sanjay PopatFrank GriesingerDepartment of Medical Oncology, Pius-Hospital Oldenburg, Oldenburg, 26121, GermanySearch for more papers by this author, Oliver CoxGlobal Access, F Hoffmann-La Roche, Basel, 4070, SwitzerlandSearch for more papers by this author, Cormac SammonPHMR, London, NW1 8XY, UKSearch for more papers by this author, Sreeram V Ramagopalan*Author for correspondence: E-mail Address: sreeram.ramagopalan@roche.comGlobal Access, F Hoffmann-La Roche, Basel, 4070, SwitzerlandSearch for more papers by this author & Sanjay PopatLung Unit, Royal Marsden National Health Service Foundation Trust, Chelsea, London, SW3 6JJ, UKSearch for more papers by this authorPublished Online:25 Feb 2022https://doi.org/10.2217/cer-2021-0296AboutSectionsPDF/EPUB ToolsAdd to favoritesDownload CitationsTrack Citations ShareShare onFacebookTwitterLinkedInRedditEmail Keywords: HTAhealth technology assessmentreal-world evidencereimbursementsingle-arm trialsHealth Technology Assessment (HTA) involves the synthesis of a broad body of clinical, humanistic and economic evidence in order to determine the relative benefit of health technologies. Guidelines set out by HTA bodies seek to provide general guidance on the methodological standards for evidence to be submitted for their consideration, and some insight into the relative value that will be assigned to different types of evidence during decision-making. The extent to which specific pieces of evidence influence HTA decisions appears to be subjectively determined by the relationship between the body of evidence available for a specific technology and the values, preferences and constraints of a given HTA body [1]. As such, a single piece of evidence might influence decision making for a single product very differently across different HTA bodies and the same type of evidence may impact the assessment of different technologies by the same HTA body differentially. Given these nuances, alongside organizational experience, manufacturers must rely not only on formal guidelines to support their decision making around evidence generation activities, but also on critical review of the outputs of HTAs carried out for other products in a similar therapeutic area and/or with a comparable evidence base. It is therefore important that clarity regarding the relative value of different pieces of evidence to the decision-making process is provided in all of these outputs.Where a relatively new type of evidence comes into use within HTA, the lack of transparency around its role in decision making processes overall and relative to other types of evidence is perhaps even more of an issue. This is currently the case for the use of real-world evidence (RWE) to estimate treatment effects as, while the potential of RWE to provide treatment effects in HTA have been purported, there remains limited detail in policies regarding the role of such RWE in HTA decision-making. Manufacturers have therefore increasingly looked to published HTA outputs to gain some insight into this. However, as has been recently highlighted [2], the contribution of RWE on treatment effects to HTA decisions in many assessments is unclear. In some cases, this obscurity can be stark, with assessment reports deficient in any commentary on the submitted RWE, as was the case for a real-world comparator arm submitted as part of the assessment of alectinib in second-line anaplastic lymphoma kinase (ALK) + non-small-cell lung cancer by the Australian Pharmaceutical Benefits Advisory Committee (PBAC) [3]. More commonly, the limitations associated with RWE have been broadly noted in the commentary of the assessments, but a detailed critique of the RWE, and its contribution to the decision has been absent. This was the case for example in the assessments of ocrelizumab in relapsing-remitting multiple sclerosis and tocilizumab in giant cell arteritis by the PBAC, the National Institute for Health and Care Excellence (NICE) and the Canadian Agency for Drugs and Technologies in Health (CADTH) [4–9]. The opaqueness in these examples may be related to the fact that in each of them RWE was used to inform secondary treatment effects of relatively minor importance to the decision problem. However, cases such as these nonetheless leave a lack of clarity regarding the value of generating supportive RWE of this nature for HTA submissions.In cases where RWE has informed a treatment effect that is pivotal to the decision problem, greater transparency has been forthcoming in HTA reports. For example, in assessments of alectinib, blinatumomab and polatuzumab vedotin, CADTH provided relatively thorough critiques of the treatment effects estimated utilizing RWE but it is difficult to determine the role the evidence played in their decisions. In contrast, in their assessment of alectinib in second line ALK+ non-small-cell lung cancer the German Institute for Quality and Efficiency in HealthCare (IQWiG) provided detailed critiques of the analyses utilizing real world comparator arms and left little ambiguity that these data could not be used to support decision making [10]. Similarly, the Haute Autorité de Santé and Norwegian Medicines agency provided similar feedback on a comparator arm including real-world patients included in submissions for tisagenlecleucel in relapsed/refractory diffuse large B cell lymphoma [11,12]. While the clarity provided in these latter cases is notable and welcome, the critiques typically point to a number of issues inherent to RWE on treatment effects, such as the potential for unmeasured confounding, and do not offer any suggestions as to a preferred way they could have been overcome using RWE. This, combined with the lack of guidelines from HTA bodies on methods for the generation of such evidence, results in inconsistency and again makes understanding the benefit of generating RWE for use in HTA submissions unclear.Interestingly, even within the HTA re-assessment setting, where the potential of RWE on treatment effects to add value is potentially highest, and collection of real-world data are often mandated, its role remains obscure. For example, while the use of observational data from the systemic anticancer therapy dataset was reported to be a key element of the NICE cancer drugs fund its use in revaluating therapies used under the cancer drugs fund was not fully transparent and appears to be very limited at best [13].We appreciate that in some of the cases we have highlighted the manufacturer(s) involved in a HTA process may have obtained further transparency on the role of RWE within the process and potential alternative options, both verbally within meetings or within unpublished documents and correspondence. However, if this is the case, we believe there would be substantial gain in publishing any such insights within the main HTA outputs in order to allow a wider set of stakeholders to benefit from them.We also appreciate that the lack of detailed feedback in some of these cases is likely to reflect the absence of straightforward solutions to the challenges encountered in using RWE in this setting. As such, we would emphasize that the responsibility to define best practices does not lie with HTA bodies alone and that a collaborative effort including regulators, payers/HTA bodies, manufacturers, academic methodologists and healthcare professionals is needed to develop and validate approaches to overcome these challenges. For example, with an appropriate framework for their use, methods such as quantitative bias analysis may provide a tool to support the assessment of submissions using RWE in the context of their potential biases [14]. While guideline and process documents updates will serve as the primary guide for best practice in using RWE in HTA, these tend to be relatively static documents and can lack the granularity to address the nuances of specific technologies. We therefore believe that, even with the development of guidelines, HTA assessment outputs represent an important opportunity to provide detailed insights into specific use cases and the latest opinions held by a HTA body regarding best practices.Importantly, the potential implications of this observation extend beyond resource allocation issues to ethical considerations. As others have recently noted [15], with the role of RWE in decision-making being relatively opaque, the benefit of generating RWE is unclear, a situation which calls into question the ethics of utilizing patient data to generate such evidence. Ethical concerns can pose a tangible barrier to recruitment of patients into individual real-world studies and, perhaps more importantly for RWE, can contribute negatively to the debate regarding the set-up and use of comprehensive linked databases of patient healthcare data, as was seen with the failure of the Care.Data initiative in England [16]. As such, it is vital that any role for RWE in supporting patient access to medicines is clarified in order to enhance patient consent for the sharing and use of their RWD and ultimately drive the development of better quality RWE for HTA.In conclusion, while it can be challenging to communicate the role of a specific piece of evidence plays in a complex decision-making process, we believe there is a need for greater clarity regarding the role of RWE in HTA. This clarity will primarily be provided in updated methodological processes and guidelines. However, any such guidelines are unlikely to remain up to date and to address all the nuances encountered in specific HTA submissions. Outputs describing the HTA assessment of specific technologies represent an important and complementary forum which HTA bodies should utilize to provide further clarity regarding the role of RWE in HTA and evolving expectations regarding its use.Financial & competing interests disclosureO Cox and SV Ramagopalan are employees of F. Hoffmann-La Roche. S Popat is a consultant to Amgen, AstraZeneca, Bayer, Beigene, Blueprint, Boehringer Ingelheim, Daiichi Sankyo, GSK, Guardant Health, Incyte, Janssen, Eli Lilly, Merck KGaA, Novartis, Roche, Takeda, Pfizer, Seattle Genetics, Turning Point Therapeutics, Xcovery outside of the submitted work. F Griesinger has consulted or provided expert opinion for Amgen, AstraZeneca, Bayer, BMS, Boehringer Ingelheim, Celgene, GSK, Lilly, MSD, Novartis, Pfizer, Roche, Siemens, and Takeda outside of the submitted work. The authors have no other relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript apart from those disclosed.No writing assistance was utilized in the production of this manuscript.Open accessThis work is licensed under the Attribution-NonCommercial-NoDerivatives 4.0 Unported License. To view a copy of this license, visit http://creativecommons.org/licenses/by-nc-nd/4.0/.