Health Technology Assessment Committee Research Articles

OP57 Consumer Engagement In Health Technology Assessment: An Australian Journey To Date

IntroductionAustralian health technology assessment (HTA) committees and processes at the national level have needed to consider an increasing focus on patient involvement and interactions within their scope. As various approaches have been explored, the visibility and impacts of patient involvement and consumer representation has evolved markedly over the most recent five years.MethodsProgram activities were developed from review of contemporary HTA models of practice across various HTA sectors internationally alongside input from individual patients and patient groups. Practical application of program pilots was influenced by specific requirements of the Office of HTA, the coordination of HTA Committee procedures and timelines, and involvement of HTA Committee consumer members.ResultsThe development of capacity building programs for HTA consumer committee members, the growth of external activity and communications targeting patient groups and networks, and the establishment of an expert position to develop the HTA Consumer Evidence and Engagement Unit within the Department’s Office of HTA, are all examples of the work delivered in this period to better integrate patient evidence and inputs into HTA processes and decision-making. Activities over the next period will seek to establish formal frameworks for meaningful involvement that can demonstrate these contributions to evaluations and decision-making.ConclusionsVarious examples of patient involvement and participation in processes have evolved over time. The scope and impacts of these experiences have illustrated important participatory demands and resourcing needs, as well potential benefits for deliberations. The Australian Government and Departmental frameworks for HTA currently seek to enhance the development of patient evidence inputs and participation. These developments will be informed by the Australian context and comprehensive consultation with Australian patient groups and their networks. Strategies for improvements are anticipated to be relevant across all HTA processes and Committees within HTA areas.

PP145 Improving Patient Expert Involvement In The Lifecycle Of Health Technology Assessments To Build Public Confidence In Decision-Making

IntroductionInvolving patients in the health technology assessment (HTA) lifecycle is a core principle at the National Institute for Health and Care Excellence (NICE). We include both patient organizations and patient experts, which helps build public confidence in healthcare decision-making. We continually work with patient experts to improve their experience and ability to participate by seeking patient expert feedback after every committee meeting.MethodsWe sent patient experts an anonymous experience survey containing a five-point Likert scale and open text boxes to capture qualitative data. The survey covered their overall experience, interaction with the committee Chair, and the support they received from both NICE and the Public Involvement Programme (PIP).In the 2019 to 2020 period we sent out 59 questionnaires and received 29 responses (47%), all of which were from medicines HTA committee participants. In the 2020 to 2021 period we sent out 120 questionnaires and received 65 responses (54%), of which 64 were from patient experts who attended medicines HTAs and one was from a medical devices HTA committee participant.ResultsGood or excellent experiences were reported by 90 percent of patient experts. The four main success factors noted were: good support before meetings; being welcomed and respected; well organized meetings; and patient expert input being valued. Areas for further improvement included: providing better briefing before meetings; allowing more time to review documents; providing more technical support; and giving more consideration to the opinions of patient experts.ConclusionsAs a result of the feedback received, the PIP now holds monthly group briefing meetings for patient experts. We also publish the anonymized feedback from the patient experts quarterly in a newsletter for committee members and share the data with internal NICE teams. Additionally, NICE aims to: send committee papers out earlier; have the option of holding a technical engagement call before committee meetings; and develop a feedback mechanism to ascertain the impact of patient input.

PD53 Improving Patient Involvement In The National Institute For Health And Care Excellence’s Appeals Process

IntroductionInvolving patients in the health technology assessment (HTA) lifecycle is a core principle at the National Institute for Health and Care Excellence (NICE) that helps build public confidence in healthcare decision-making. The last stage in this lifecycle is an appeal whereby stakeholders, including patient organizations, can appeal against the HTA committee’s decision based on two specific grounds: (i) the committee has failed to act fairly or NICE has exceeded its powers; and (ii) the recommendation is unreasonable considering the evidence submitted to NICE. Improving patient involvement in the appeals process was identified as a key area from consultative work undertaken with patient organizations in 2019.MethodsIn September 2020, surveys were sent to patient organizations for their feedback. The organizations received a survey tailored to their circumstances in relation to the following four outcomes.(i)An appeal was lodged that: (a) resulted in an appeal hearing; or (b) did not result in an appeal hearing.(ii)An appeal was not lodged but the organization had received: (a) negative guidance; or (b) ‘optimized’ guidance.ResultsSixteen responses were received across the four surveys. The key findings in priority order were as follows.(i)The legalist nature of appeal hearings is off-putting.(ii)The online appeals guide and template letter are not easy to use.(iii)Organizations would like more information on appeal hearings and timelines.(iv)Organizations would like patient-friendly materials and training on par with the support they receive for other HTA participation.ConclusionsAs a result of these findings, we are iteratively developing materials to support patient organizations, including:•a glossary;•a redesigned appeals web page with clearer timelines and a template letter;•a ‘Frequently Asked Questions’ document specifically for patient organizations; and•a video podcast from a lay appeals panel Chair explaining appeals in a patient-friendly way.We are also including a section on appeals in our introductory training for all patient organizations participating in medicines HTAs.

OP24 Impact Of Patient Input On Cancer Drug Funding Recommendations In Canada

IntroductionPatient involvement in health technology assessment (HTA) has documented advantages, such as improved understanding of disease context, and increased legitimacy and transparency of the HTA process. In the absence of clear metrics, thresholds, or criteria, it is not clear how input regarding patient preferences influences HTA based recommendations of the pan Canadian Oncology Drug Review (pCODR).MethodsThis is a concurrent complementary mixed methods study. A quantitative model (logit) is used to estimate the impact of patient input and other HTA criteria on pCODR recommendations. A qualitative analysis of semi-structured interviews with Canadian HTA committee members is used to describe the mechanisms of action through which patient input influences recommendations.ResultsPatient input was considered important in providing context to the HTA discussion, but committee members were not able to explicate how any specific elements of patient submissions weighted into the committee’s recommendation. There was an element of mistrust in the patient input data. The estimated impact of patient input on funding recommendations is not statistically significant, recommendations remain driven by evidence of clinical benefit.ConclusionsThe commitment to inclusion of patient perspectives in HTA in Canada is strong, and procedurally Canada is among the leaders in this regard. The tangible impact of patient input could be increased with an improved system for collection of most relevant data, and clear guidelines about how patient input should weigh into HTA recommendations.

HTA235 Time to Health Technology Assessment of New Medicinal Products in Greece

The purpose of the present study was to evaluate the time elapsed between Health Technology Assessment (HTA) application to completion of the clinical evaluation of new medicinal products (including orphans) in Greece, from July 2018 (establishment of the Greek HTA Committee) until April 2022.

Estimation of Value-Based Price for Five High-Technology Medical Devices Approved by a Regional Health Technology Assessment Committee in Italy.

Background and objectivesValue-based pricing (VBP) is used quite frequently for medicines, but its application to medical devices is very limited. The objective of the present study was to conduct a pilot experience of systematic estimation of the value-based price of medical devices from the perspective of our national health system. Our experience was focused on high-technology devices (class IIb/III and active implantable). The objective was to evaluate the applicability of VBP in a real-world setting and to estimate the value-based price of devices in all cases where this estimation was feasible.MethodsThe dataset analysed in this work consists of 24 new devices approved consecutively in the Tuscany region over the period from January 2020 to December 2021. Since the calculation of value-based price requires the availability of a cost-effectiveness analysis, we searched for this information for each of these devices. The Cost-Effectiveness Analysis (CEA) Registry of Tufts Medical Center (US) and the health technology assessment (HTA) reports of our region were considered adequate sources of these data. Standard equations of cost-effectiveness were applied to determine the value-based price for these devices, and these prices were compared with the corresponding real prices charged in our region.ResultsWe found adequate information for five devices (21%) out of the total of 24. In three of these cases, the published analysis taken as a reference was based on Markov modelling. The comparison between value-based prices and real prices generally showed an acceptable concordance, though with a couple of outliers. An important finding is that, in a large proportion of cases (79%), the information needed for this calculation was lacking.ConclusionTo our knowledge, this is the first experience in which an institution of the healthcare system has tried a systematic application of VBP in the field of high-technology devices. Our results are encouraging and suggest a wider application of cost-effectiveness in this field.

Health Technology Assessment in Tanzania: Capacity and experience of HTA committee members

Background Tanzania established a Health Technology Assessment Committee in recognition of the increasing role of Health Technology Assessment (HTA) to achieve universal health coverage. The committee has the mandate to develop recommendations on the reimbursement of technologies for decision-makers. It is widely recognized that there are challenges to the technical capacity and human resource availability of HTA, including that of decision-making committees. Committee members need to combine scientific and social judgments vis-a-vis the technology under evaluation. This paper reports on a self-administered survey among committee members to assess their capacity for the tasks at hand. Methods A survey was developed and sent to all members of the HTA committee in Tanzania. Responses were analyzed, and key themes explored. Results Out of 20 committee members, 15 members completed the survey (response rate 75%). Committee members were primarily government employees who were qualified and well-experienced in clinical medicine. However, they were less qualified and experienced in HTA and expressed low confidence in this subject matter. Conclusions The results indicate the type of evidence an advisory committee can reasonably be expected to understand and handle. It can also guide future capacity-strengthening activities and programmes, and serve as a baseline for measuring the impact of these programmes.

Introducing an Ethics Framework for health priority-setting in South Africa on the path to universal health coverage

Background. South Africa (SA) has embarked on a process to implement universal health coverage (UHC) funded by National Health Insurance (NHI). The 2019 NHI Bill proposes creation of a health technology assessment (HTA) body to inform decisions about which interventions NHI funds will cover under UHC. In practice, HTA often relies mainly on economic evaluations of cost-effectiveness and budget impact, with less attention to the systematic, specific consideration of important social, organisational and ethical impacts of the health technology in question. In this context, the South African Values and Ethics for Universal Health Coverage (SAVE-UHC) research project recognised an opportunity to help shape the health priority-setting process by providing a way to take account of multiple, ethically relevant considerations that reflect SA values. The SAVE-UHC Research Team developed and tested an SA-specific Ethics Framework for HTA assessment and analysis. Objectives. To develop and test an Ethics Framework for use in the SA context for health priority-setting. Methods. The Framework was developed iteratively by the authors and a multidisciplinary panel (18 participants) over a period of 18 months, using the principles outlined in the 2015 NHI White Paper as a starting point. The provisional Ethics Framework was then tested with multi-stakeholder simulated appraisal committees (SACs) in three provinces. The membership of each SAC roughly reflected the composition of a potential SA HTA committee. The deliberations and dedicated focus group discussions after each SAC meeting were recorded, analysed and used to refine the Framework, which was presented to the Working Group for review, comment and final approval. Results. This article describes the 12 domains of the Framework. The first four (Burden of the Health Condition, Expected Health Benefits and Harms, Cost-Effectiveness Analysis, and Budget Impact) are commonly used in HTA assessments, and a further eight cover the other ethical domains. These are Equity, Respect and Dignity, Impacts on Personal Financial Situation, Forming and Maintaining Important Personal Relationships, Ease of Suffering, Impact on Safety and Security, Solidarity and Social Cohesion, and Systems Factors and Constraints. In each domain are questions and prompts to enable use of the Framework by both analysts and assessors. Issues that arose, such as weighting of the domains and the availability of SA evidence, were discussed by the SACs. Conclusions. The Ethics Framework is intended for use in priority-setting within an HTA process. The Framework was well accepted by a diverse group of stakeholders. The final version will be a useful tool not only for HTA and other priority-setting processes in SA, but also for future efforts to create HTA methods in SA and elsewhere.

Mentoring as a potential means to assist consumer representatives contribute effectively to the assessment of health technologies in Australia: a pragmatic review.

This project was implemented on behalf of the Health Technology Assessment (HTA) Consumer Consultative Committee (CCC) to explore training to support and retain new consumer representatives to participate effectively in HTA committees. These committees are key parts of the Australian Government's health system. Currently, there is no training available to them, specific to their roles in HTA committees. Hypothesizing that mentoring is appropriate, the project team undertook a literature review to identify definitions of mentoring, its benefits, skills requirements, resources, examples of best practice, and how mentoring might support consumer representatives in formal health technology assessment committee structures. A rapid review was commenced by the project team and fifty-seven articles were identified and read independently. Following discussion, the team revised its approach as there was little evidence to assess and drew upon thirty-five articles where elements of mentoring were described. Discussion was followed by a thematic qualitative analysis exploring mentoring models. The project team agreed that features of mentoring programs were necessary to design a mentoring program under the headings of definitions, mentors' qualities, benefits of training, resources, other considerations, and evaluation. These assist the design of a pilot project to test mentoring's effectiveness. Mentoring may assist consumers working in the health technology area to develop their skills and competencies and contribute to representing the needs of health consumers in the approval of applications. A pilot mentoring program is currently being designed and will run with one mentor and one mentee in an HTA committee.

Similarities and Differences in Health Technology Assessment Systems and Implications for Coverage Decisions: Evidence from 32 Countries.

Health technology assessment (HTA) systems across countries vary in the way they are set up, according to their role and based on how funding decisions are reached. Our objective was to study the characteristics of these systems and their likely impact on the funding of technologies undergoing HTA. Based on a literature review, we created a conceptual framework that captures key operating features of HTA systems. We used this framework to map current HTA activities across 32 countries in the European Union, the UK, Canada and Australia. Evidence was collected through a systematic search of competent authority websites and grey literature sources. Primary data collection through expert consultation validated our findings and further complemented the analysis. Sixty-three HTA bodies were identified. Most have a national scope (76%), are independent (73%), have an advisory role (52%), evaluate pharmaceuticals predominantly or exclusively (76%), assess health technologies based on their clinical and cost-effectiveness (73%) and involve various stakeholders as members of the HTA committee (94%) and/or through external consultation (76%). The majority of HTA outcomes are not legally binding (81%). Although all study countries implement HTA, the way it fits into decision-making, negotiation processes, and coverage and funding decisions differs significantly across countries. HTA is a dynamic and transformative process and there is a need for transparency to investigate whether evidence-based information influences coverage decisions.Supplementary InformationThe online version contains supplementary material available at 10.1007/s41669-021-00311-5.

Parametric Survival Extrapolation of Early Survival Data in Economic Analyses: A Comparison of Projected Versus Observed Updated Survival

ObjectivesTo establish the value of cancer drugs by cost-effectiveness analysis, lifetime parametric survival extrapolations are often fitted to early data. Recent literature suggests that the benefit of cancer agents in primary publications is often different compared with updated data. This study aimed to examine the projected survival based on parametric extrapolations compared with observed survival based on updated data. MethodsUS Food and Drug Administration oncology approvals from January 2006 to December 2015 were reviewed to identify randomized controlled trials, with updated overall survival (OS) or progression-free survival (PFS) data within 5 years. Individual patient data were reconstructed using established methods on initial and updated publications. Projected survival was calculated as the best-fit parametric restricted mean survival time (RMST) based on extrapolated initial Kaplan-Meier curves whereas observed survival was calculated as observed RMST based on updated Kaplan-Meier curves. Mean deviations, mean absolute error (MAE), mean absolute percentage error, and linear regressions were conducted to examine the relationship between projected and observed survival. ResultsIn total, 32 randomized controlled trials were included. The MAE between the projected RMST and observed RMST was 3.18 months (OS) and 2.84 months (PFS) and absolute percentage error of 100% (OS) and 23% (PFS), suggesting substantial imprecision of the projected RMST in predicting the updated RMST. The linear regression indicated MAE increased as time extrapolated and as the percentage of censored patients increased. ConclusionsThis study demonstrated substantial difference in projected survival between initial and updated publications. Health technology assessment committees need to be aware of the potential uncertainty of incremental effectiveness and resultant value-for-money assessment when making reimbursement decisions based on initial publications with immature survival data.

Empirical Estimates of the Marginal Cost of Health Produced by a Healthcare System: Methodological Considerations from Country-Level Estimates.

Many health technology assessment committees have an explicit or implicit reference value (often referred to as a ‘threshold’) below which new health technologies or interventions are considered value for money. The basis for these reference values is unclear but one argument is that it should be based on the health opportunity costs of funding decisions. Empirical estimates of the marginal cost per unit of health produced by a healthcare system have been proposed to capture the health opportunity costs of new funding decisions. Based on a systematic search, we identified eight studies that have sought to estimate a reference value through empirical estimation of the marginal cost per unit of health produced by a healthcare system for England, Spain, Australia, The Netherlands, Sweden, South Africa and China. We review these eight studies to provide an overview of the key methodological approaches taken to estimate the marginal cost per unit of health produced by the healthcare system with the aim to help inform future estimates for additional countries. The lead author for each of these papers was invited to contribute to the current paper to ensure all the key methodological issues encountered were appropriately captured. These included consideration of the key variables required and their measurement, accounting for endogeneity of spending to health outcomes, the inclusion of lagged spending, discounting and future costs, the use of analytical weights, level of disease aggregation, expected duration of health gains, and modelling approaches to estimating mortality and morbidity effects of health spending. Subsequent research estimates for additional countries should (1) carefully consider the specific context and data available, (2) clearly and transparently report the assumptions made and include stakeholder perspectives on their appropriateness and acceptability, and (3) assess the sensitivity of the preferred central estimate to these assumptions.Supplementary InformationThe online version contains supplementary material available at 10.1007/s40273-021-01087-6.

Uncertainty tolerance among experts involved in drug reimbursement recommendations: Qualitative evidence from HTA committees in Canada and Poland

ObjectivesDrug reimbursement decisions often rely on health technology assessment (HTA). Increasingly, new drugs have limited clinical evidence and uncertain clinical benefit. Our goal was to describe how members of drug advisory committees and other stakeholders conceptualize and tolerate uncertainty and how they rationalize uncertainty tolerance. MethodsOur triangulated parallel design applied two qualitative methods. We interviewed 31 members of drug advisory committees in Canada and Poland about their information needs and included hypothetical scenarios with uncertain clinical benefits. Respondents speculated about their likely reimbursement recommendation. We analyzed written recommendations of the pan Canadian Oncology Drug Review for drugs with uncertain benefit and compared initial recommendations to the responses from patient and clinician groups. ResultsUncertainty tolerance varied among committee members and across jurisdictions. In the scenario analysis, 7 Canadian and 11 Polish respondents leaned against recommending a hypothetical drug with uncertain clinical benefit, whereas 5 Canadian and 5 Polish respondents leaned in favour. Those against rationalized that uncertainty increases potential harm; those in favour rationalized that patients often have no alternatives. The document analysis revealed that patients had higher uncertainty tolerance in general. ConclusionsUncertainty tolerance varies among committee members and other stakeholders depending on their backgrounds and on the decision contexts. We argue that policy guidance around uncertainty management could improve the transparency and consistency of recommendations.

Quality Assessment of HTA reports: a Critical Appraisal Toolkit within an institutional HTA program

Abstract Issue With population aging, increased demand for healthcare and limited availability of economic and human resources, well-functioning and sustainable health systems have to rely on rigorous and evidence-based assessments of complex technologies' clinical effectiveness and safety. The Lombardy Region in Northern Italy has a well-established HTA program that offers technical support to its hospitals' network to produce and to review Health Technology Assessment (HTA) reports. Description of the Problem To better coordinate the HTA network and to reduce self-referencing of University and Community hospitals, Lombardy Region carried out an intense field training and distance learning from year 2017, with a project aimed at involving health care professionals in an accurate evaluation of technologies in 18 HTA reports. The regional HTA Supporting Centre developed a Toolkit for the critical appraisal of reports and supporting literature to improve the quality of hospital-based reports. Results In compliance with the regional Law DGR XI/1046 17.12.2018 and the framework proposed, during year 2019 hospitals used the Toolkit to help writing complete and good quality HTA reports on 37 different technologies. With a Public Health resident internship, the Toolkit was revised, extended and then used in 2020 by the hospital's HTA network during the double blinded peer review, mandated by the regional Law DGR XI/2672 16.12.2019, of the year 2019 HTA reports. Detailed results will be presented at the conference. Lessons We observed an increase in number of HTA reports from 2018 to 2019, a greater language's and format's homogeneity and an improvement in the quality of some reports, which will be submitted to the Regional HTA Committee for appraisal and reimbursement. Key messages The development and use of a Critical Appraisal Toolkit in an Regional HTA program can help hospitals write complete and good quality HTA reports. A centrally supported quality improvement of distributed assessment activities in a network of hospitals can enhance the production of HTA reports, relevant to the needs of a local healthcare system.

Evaluation of the impact of patient involvement in health technology assessments: A scoping review.

While involving patients in health technology assessment (HTA) has become increasingly common and important around the world, little is known about the optimal methods of evaluating patients' involvement (PI) in HTA. This scoping review was undertaken to provide an overview of currently available methods for the evaluation of PI, specifically the impact of PI on HTA recommendations. A literature search was conducted using nine databases as well as a grey literature search of the websites of 26 organizations related to the conduct, practice or research of HTA to identify articles, reports and abstracts related to the evaluation of PI impact in HTA. We identified 1,248 unique citations, six of which met our eligibility criteria. These six records (five articles, and one report) were all published after 2012. Four assessed the impact of patient experience submissions on final HTA recommendations; one evaluated the impact of direct involvement on HTA committees, and one assessed impact of multiple forms of involvement. Methods of evaluation included quantitative analyses of reimbursement decisions, qualitative interviews with those directly involved in an assessment, surveys of patient groups and committee members, and the review of HTA reports. Quantitative evaluation of PI based on associations with funding decisions may not be feasible or fully capture the relevant impact of PI in the assessment of health technologies. Rather, a combination of both qualitative and quantitative strategies may allow for the most comprehensive assessment of the impact of PI on HTA recommendations when possible.

Progression criteria in trials with an internal pilot: an audit of publicly funded randomised controlled trials

BackgroundWith millions of pounds spent annually on medical research in the UK, it is important that studies are spending funds wisely. Internal pilots offer the chance to stop a trial early if it becomes apparent that the study will not be able to recruit enough patients to show whether an intervention is clinically effective. This study aims to assess the use of internal pilots in individually randomised controlled trials funded by the Health Technology Assessment (HTA) programme and to summarise the progression criteria chosen in these trials.MethodsStudies were identified from reports of the HTA committees’ funding decisions from 2012 to 2016. In total, 242 trials were identified of which 134 were eligible to be included in the audit. Protocols for the eligible studies were located on the NIHR Journals website, and if protocols were not available online then study managers were contacted to provide information.ResultsOver two-thirds (72.4%) of studies said in their protocol that they would include an internal pilot phase for their study and 37.8% of studies without an internal pilot had done an external pilot study to assess the feasibility of the full study. A typical study with an internal pilot has a target sample size of 510 over 24 months and aims to recruit one-fifth of their total target sample size within the first one-third of their recruitment time.There has been an increase in studies adopting a three-tiered structure for their progression rules in recent years, with 61.5% (16/26) of studies using the system in 2016 compared to just 11.8% (2/17) in 2015. There was also a rise in the number of studies giving a target recruitment rate in their progression criteria: 42.3% (11/26) in 2016 compared to 35.3% (6/17) in 2015.ConclusionsProgression criteria for an internal pilot are usually well specified but targets vary widely. For the actual criteria, red/amber/green systems have increased in popularity in recent years. Trials should justify the targets they have set, especially where targets are low.

PMU74 HEALTH TECHNOLOGY ASSESSMENT IN BULGARIA – 3 YEAR REPORT CARD

A designated Health Technology Assessment (HTA) body was introduced in Bulgaria in December 2015, as part of the National Centre for Public Health and Analyses (NCPHA). The assessment process involves a weighted point-based scoring system covering 11 key criteria, including efficacy, safety, QoL, cost-effectiveness, and budget impact. A therapy scoring 75% of the maximum 965 points can be recommended for addition to the Positive Drug List (PDL), while a score between 50% and 75% allows PDL access with specific restrictions (e.g. specific populations only). This research evaluates the impact of the Bulgarian HTA committee. All publicly-available HTA assessments issued by the Bulgarian HTA committee (01/01/2015-31/12/2018) were screened and key information extracted. 75 HTA reports were published on the NCPHA website. 68/75 (91%) of the assessments received a positive recommendation for PDL addition, including 47/75 (63%) receiving unconditional positive recommendations (scoring ≥75% in the scoring criteria). Only 7/75 (9%) received a negative opinion. 24/75 (32%) of the assessments were in oncology, 21/24 (88%) of which received a positive recommendation (46% unconditionally). Many reports state that an incremental cost-effectiveness threshold of 37,000BGN ($21,700USD) per QALY applies in the assessment process. The Bulgarian healthcare system is taking major strides in the direction of maximising the opportunity to offer cost-effective healthcare in a systematic approach. Further, from 2018, only cost-effective medicines with a positive recommendation and offered at a confidential discount to Bulgaria’s National Health Insurance Fund (NZOK) will be reimbursed, eliminating the prior costly practice of covering a basic threshold for a multitude of drugs, with patient co-pay. With almost two thirds of assessed therapies receiving unconditional recommendations to date, this HTA process holds promise to both relieve pharmaceutical expenditure, which has put immense pressure on NZOK in recent years, whilst maintaining access to innovative therapies.

Multicriteria decision analysis (MCDA) for health technology assessment: the Queensland Health experience.

Objectives In determining whether new health technologies should be funded, health technology assessment (HTA) committees prefer explicit to implicit methods of analysis in enhancing transparency and consistency of decision making. The aim of this study was to develop and pilot a multicriteria decision analysis (MCDA) framework for the Queensland Department of Health HTA program committee, which weighted decision making criteria according to their perceived importance as determined by group consensus. Methods The criteria used in the MCDA framework were identified by reviewing the five unweighted criteria used in the existing process, consultation with committee members and literature review. Criteria were clearly defined and ordinal categories of lowest to highest preferred were assigned against which technology submissions would be rated. Criteria weights were determined through a discrete choice experiment (DCE) survey of committee members using validated software. Mean weighted technology scores were then used to guide deliberative discussions in determining final funding decisions. Results The MCDA framework created one additional criterion to the previous five. The criteria and their mean weights identified through the DCE survey were clinical benefit and safety (27.2%), quality of evidence (19.2%), implementation capacity (16.9%), innovation (15.4%), burden of disease and clinical need (13.3%) and societal and ethical values (8.0%). Criterion weights varied considerably between individual committee members, with one criterion having a difference of 36.9% between the highest and lowest preference weights. Following deliberative discussions, all but one of 10 submissions were awarded funding. The submission not supported received the third lowest score through the MCDA model. Conclusions This pilot application of an MCDA framework, as a complement to committee deliberation, conferred greater transparency and objectivity on HTA assessment of technologies. The framework converted an implicit, unweighted review process to one that is more explicit, flexible in weighting importance and pragmatic. What is known about the topic? HTA programs involve complex decision-making processes requiring the consideration of multiple criteria. Explicit methods of analysis that use weighted criteria according to their relative importance enhance transparency and consistency of decision making by HTA committees, and are preferred to implicit reviews using unweighted criteria. What does this paper add? This article describes the development and piloting of an MCDA framework that aims to improve transparency, objectivity and consistency of funding decisions of the Queensland HTA committee. Criteria were identified through a review of current processes, committee discussions and a literature review, and the Grading of Recommendations Assessment, Development and Evaluation (GRADE) quality of evidence system. Criteria were weighted using a discrete choice experiment involving committee members. Using weighted criteria, mean technology scores were calculated and incorporated into deliberative discussions to determine funding decisions. What are the implications for practitioners? The MCDA framework described here converted a more implicit, unweighted process to one that was more pragmatic, explicit and flexible in scoring HTA submissions. This framework may be useful to other HTA programs and could be expanded to resource allocation decision making in many other healthcare settings.

Patient advocate perspectives on involvement in HTA: an international snapshot

Plain English summaryA number of health technology assessment (HTA) organisations have developed processes to engage patients in the assessment of new health technologies such as pharmaceuticals, diagnostic tests, devices or medical procedures. Typically, this involves the HTA agency providing an opportunity for patient advocates and their patient organisations (support groups for patients with a specific disease or condition) to provide submissions detailing experiences with the disease and the health technology that is being assessed. While some literature exists about how HTA agencies view the engagement of patients in the HTA process, it is not yet clear how the patient advocates and patient organisations themselves view this engagement. To answer this question, we surveyed the views of patient advocates who were members of patient organisations known to be engaged in the process of HTA or evidence-based practice. Snowballing – that is, passing on the survey invitation from individuals invited to take part in the survey to other individuals – occurred in one of the countries. The responses in this country provided a very useful comparison between the views of people who were appointed as the ‘patient representatives’ on an HTA committee with those who contributed input as part of the general patient organisation engagement process. Our findings identify gaps in understanding of the purpose of patient involvement and whether patient organisations felt their input made a difference, the information and support provided, and if and how feedback is given to the patient organisations. Our work can help inform further research as well as continuing improvements in HTA patient engagement processes.BackgroundPatient involvement in health technology assessment (HTA) processes is becoming more frequent. However, it is not clear how patient advocates and their disease-based patient organisations that are involved in HTA view their involvement. We report on the results of an international survey of patient advocates and members of patient organisations about their experiences and perceptions of that involvement.MethodsA 16-question survey was sent out to patient advocates and members of patient groups known to be involved in HTA processes or evidence-based practice. The survey consisted of open-ended questions focusing on respondent characteristics, stage and nature of involvement, support from HTA agencies for involvement, purpose of involvement, feedback on involvement, and whether the respondents felt that their input made a difference.ResultsOf 16 individuals who received the survey, 15 responded. Three, from Italy, Israel and Japan, were not involved in HTA in their country. Respondents from the following countries reported involvement in HTA processes: Canada, England, Scotland, and Wales, The Netherlands, Australia, Taiwan. The respondents indicated that HTA agencies reach out to them either actively or passively, and that their involvement is often at the appraisal stage of HTA. Typically, they reported involvement as either participants in committees or providers of submissions to HTA agencies. A wide range of approaches to supporting patient involvement by the HTA agencies was identified by respondents – including personal and telephone support, online resources, training and provision of information – but the level and type of support reported was uneven across jurisdictions. Not all respondents were clear on the purpose of their involvement in HTA, although some were able to cite specific examples of how their input made a difference; members of an HTA decision-making committee appeared to have a better understanding and were able to give examples. Feedback from HTA agencies to the patient groups on their submissions is often not provided.ConclusionsAlthough considerable progress has been made in terms of engaging patients and patient groups in HTA, gaps remain in how involvement is supported, including facilitating involvement, clarity on roles, two-way flow of information, and methods for enhancing communication between patient organisations and HTA agencies.

PP071 Health Technology Assessment In Bulgaria: A Review Of The First Fifteen Reports Assessed

INTRODUCTION:In Bulgaria, the regulatory body sets for the first time legal requirements for Health Technology Assessment (HTA) in Law on Medicinal Products in Human Medicine (LMPHM) on 27 June 2015. The next essential step for HTA capacity building was the promulgation of Ordinance 9 / December 1, 2015 on the conditions and procedures for conducting health technology assessment by the Ministry of Health (1). In the beginning of 2016, the Main Price and Reimbursement Committee was set and launched a process for establishing the small working groups with the task of reviewing the first applicants reports of pharmaceuticals for inclusion in the Positive Drug List (PDL).METHODS:The objective of this study is to summarize the recommendations of the newly established HTA Committee in Bulgaria and to examine the characteristics of the technologies and the key considerations that led to those decisions. We systematically read all published by the Committee recommendations for 2016 and analyzed them under: type of recommendations (positive or negative for inclusion in PDL), population, specialization, type of service, type of justification and the impact on final conclusions.RESULTS:For the first year of its work the HTA Committee was able to assess fifteen technologies (pharmaceuticals) and only one received a negative recommendation (6 percent) from the working group. All the rest (n = 14; 94 percent) were recommended for funding. The final recommendation from the Main Price and Reimbursement Committee is available for four (27 percent) technologies – all positive for inclusion in PDL. All recommendations were connected with adults and in oncology (n = 4; 27 percent); heart diseases (n = 4; 27 percent); Chronic Obstructive Pulmonary Disease, COPD (n = 2; 13 percent); diabetes (n = 2; 13 percent); psoriasis (n = 2; 13 percent); Hepatitis C (n = 1; 7 percent). The only negative recommendation was justified due to lack of robust evidence, safety issues and credibility of HTA analysis (2).CONCLUSIONS:The information about the number of applications received from the Committee is not available and correct conclusions about the capability is not possible, but indirect circumstances, as the lack of well-trained HTA experts, certainly impede establishment of the small working groups and slow the assessment process (3). At this point it is clear that additional efforts are need to overcome the barriers and smooth adoption and implementation of HTA methods in Bulgaria.