Patient and public involvement in health technology assessment committees: who is free from conflicts of interest?

Objectives In determining whether new health technologies should be funded, health technology assessment (HTA) committees prefer explicit to implicit methods of analysis in enhancing transparency and consistency of decision making. The aim of this study was to develop and pilot a multicriteria decision analysis (MCDA) framework for the Queensland Department of Health HTA program committee, which weighted decision making criteria according to their perceived importance as determined by group consensus. Methods The criteria used in the MCDA framework were identified by reviewing the five unweighted criteria used in the existing process, consultation with committee members and literature review. Criteria were clearly defined and ordinal categories of lowest to highest preferred were assigned against which technology submissions would be rated. Criteria weights were determined through a discrete choice experiment (DCE) survey of committee members using validated software. Mean weighted technology scores were then used to guide deliberative discussions in determining final funding decisions. Results The MCDA framework created one additional criterion to the previous five. The criteria and their mean weights identified through the DCE survey were clinical benefit and safety (27.2%), quality of evidence (19.2%), implementation capacity (16.9%), innovation (15.4%), burden of disease and clinical need (13.3%) and societal and ethical values (8.0%). Criterion weights varied considerably between individual committee members, with one criterion having a difference of 36.9% between the highest and lowest preference weights. Following deliberative discussions, all but one of 10 submissions were awarded funding. The submission not supported received the third lowest score through the MCDA model. Conclusions This pilot application of an MCDA framework, as a complement to committee deliberation, conferred greater transparency and objectivity on HTA assessment of technologies. The framework converted an implicit, unweighted review process to one that is more explicit, flexible in weighting importance and pragmatic. What is known about the topic? HTA programs involve complex decision-making processes requiring the consideration of multiple criteria. Explicit methods of analysis that use weighted criteria according to their relative importance enhance transparency and consistency of decision making by HTA committees, and are preferred to implicit reviews using unweighted criteria. What does this paper add? This article describes the development and piloting of an MCDA framework that aims to improve transparency, objectivity and consistency of funding decisions of the Queensland HTA committee. Criteria were identified through a review of current processes, committee discussions and a literature review, and the Grading of Recommendations Assessment, Development and Evaluation (GRADE) quality of evidence system. Criteria were weighted using a discrete choice experiment involving committee members. Using weighted criteria, mean technology scores were calculated and incorporated into deliberative discussions to determine funding decisions. What are the implications for practitioners? The MCDA framework described here converted a more implicit, unweighted process to one that was more pragmatic, explicit and flexible in scoring HTA submissions. This framework may be useful to other HTA programs and could be expanded to resource allocation decision making in many other healthcare settings.

IntroductionAustralian health technology assessment (HTA) committees and processes at the national level have needed to consider an increasing focus on patient involvement and interactions within their scope. As various approaches have been explored, the visibility and impacts of patient involvement and consumer representation has evolved markedly over the most recent five years.MethodsProgram activities were developed from review of contemporary HTA models of practice across various HTA sectors internationally alongside input from individual patients and patient groups. Practical application of program pilots was influenced by specific requirements of the Office of HTA, the coordination of HTA Committee procedures and timelines, and involvement of HTA Committee consumer members.ResultsThe development of capacity building programs for HTA consumer committee members, the growth of external activity and communications targeting patient groups and networks, and the establishment of an expert position to develop the HTA Consumer Evidence and Engagement Unit within the Department’s Office of HTA, are all examples of the work delivered in this period to better integrate patient evidence and inputs into HTA processes and decision-making. Activities over the next period will seek to establish formal frameworks for meaningful involvement that can demonstrate these contributions to evaluations and decision-making.ConclusionsVarious examples of patient involvement and participation in processes have evolved over time. The scope and impacts of these experiences have illustrated important participatory demands and resourcing needs, as well potential benefits for deliberations. The Australian Government and Departmental frameworks for HTA currently seek to enhance the development of patient evidence inputs and participation. These developments will be informed by the Australian context and comprehensive consultation with Australian patient groups and their networks. Strategies for improvements are anticipated to be relevant across all HTA processes and Committees within HTA areas.

This project was implemented on behalf of the Health Technology Assessment (HTA) Consumer Consultative Committee (CCC) to explore training to support and retain new consumer representatives to participate effectively in HTA committees. These committees are key parts of the Australian Government's health system. Currently, there is no training available to them, specific to their roles in HTA committees. Hypothesizing that mentoring is appropriate, the project team undertook a literature review to identify definitions of mentoring, its benefits, skills requirements, resources, examples of best practice, and how mentoring might support consumer representatives in formal health technology assessment committee structures. A rapid review was commenced by the project team and fifty-seven articles were identified and read independently. Following discussion, the team revised its approach as there was little evidence to assess and drew upon thirty-five articles where elements of mentoring were described. Discussion was followed by a thematic qualitative analysis exploring mentoring models. The project team agreed that features of mentoring programs were necessary to design a mentoring program under the headings of definitions, mentors' qualities, benefits of training, resources, other considerations, and evaluation. These assist the design of a pilot project to test mentoring's effectiveness. Mentoring may assist consumers working in the health technology area to develop their skills and competencies and contribute to representing the needs of health consumers in the approval of applications. A pilot mentoring program is currently being designed and will run with one mentor and one mentee in an HTA committee.

IntroductionVarious committees provide health technology assessment (HTA) recommendations to the Australian government on health technology subsidization. All committees include consumer members, but operations vary across committees. To support the committee’s consumer members and facilitate consumer input, the Consumer Evidence and Engagement Unit (CEEU) was launched in 2019. The CEEU aims to enhance inclusion of consumer input in Commonwealth HTA processes.MethodsThe CEEU operates within the Australian Government Department of Health and Aged Care in a hybrid format. The core functions of the CEEU include secretariat support for the HTA Consumer Consultative Committee (CCC), which comprises consumer members from all HTA committees. The CEEU also summarizes consumer input for the HTA committees. In 2022, the CEEU launched the Conversations for Change series, which was developed through a range of different consultation activities. As a result, an ongoing “Changing Conversations” mandate is in place, to build and sustain relational, non-transactional partnerships in a true collaborative style.ResultsThe HTA committees and subcommittees operate across cycles with overlapping timelines, requiring consumer input to be gathered at different timepoints for different purposes. The CEEU has launched materials, including the Consultation Hub and HTA Engage, and is developing a Consultation Toolkit to raise awareness and facilitate these processes. The role of the HTA CCC is evolving as a forum to enable consumer committee members to share experiences and receive CEEU support. Improvements in consultation processes have enabled the CEEU to provide consumer comments to consumer committee members to facilitate genuine representation of consumer and other external stakeholder views.ConclusionsManaging consumer input can be resource intensive, however there is increasing recognition of the importance of consumer engagement in Australian HTA. The CEEU expects continued growth and a wider remit from 2024, as new processes and changes occur. Additional materials are planned to support consumer committee members and to build capacity among consumers and consumer organizations.

Plain English summaryA number of health technology assessment (HTA) organisations have developed processes to engage patients in the assessment of new health technologies such as pharmaceuticals, diagnostic tests, devices or medical procedures. Typically, this involves the HTA agency providing an opportunity for patient advocates and their patient organisations (support groups for patients with a specific disease or condition) to provide submissions detailing experiences with the disease and the health technology that is being assessed. While some literature exists about how HTA agencies view the engagement of patients in the HTA process, it is not yet clear how the patient advocates and patient organisations themselves view this engagement. To answer this question, we surveyed the views of patient advocates who were members of patient organisations known to be engaged in the process of HTA or evidence-based practice. Snowballing – that is, passing on the survey invitation from individuals invited to take part in the survey to other individuals – occurred in one of the countries. The responses in this country provided a very useful comparison between the views of people who were appointed as the ‘patient representatives’ on an HTA committee with those who contributed input as part of the general patient organisation engagement process. Our findings identify gaps in understanding of the purpose of patient involvement and whether patient organisations felt their input made a difference, the information and support provided, and if and how feedback is given to the patient organisations. Our work can help inform further research as well as continuing improvements in HTA patient engagement processes.BackgroundPatient involvement in health technology assessment (HTA) processes is becoming more frequent. However, it is not clear how patient advocates and their disease-based patient organisations that are involved in HTA view their involvement. We report on the results of an international survey of patient advocates and members of patient organisations about their experiences and perceptions of that involvement.MethodsA 16-question survey was sent out to patient advocates and members of patient groups known to be involved in HTA processes or evidence-based practice. The survey consisted of open-ended questions focusing on respondent characteristics, stage and nature of involvement, support from HTA agencies for involvement, purpose of involvement, feedback on involvement, and whether the respondents felt that their input made a difference.ResultsOf 16 individuals who received the survey, 15 responded. Three, from Italy, Israel and Japan, were not involved in HTA in their country. Respondents from the following countries reported involvement in HTA processes: Canada, England, Scotland, and Wales, The Netherlands, Australia, Taiwan. The respondents indicated that HTA agencies reach out to them either actively or passively, and that their involvement is often at the appraisal stage of HTA. Typically, they reported involvement as either participants in committees or providers of submissions to HTA agencies. A wide range of approaches to supporting patient involvement by the HTA agencies was identified by respondents – including personal and telephone support, online resources, training and provision of information – but the level and type of support reported was uneven across jurisdictions. Not all respondents were clear on the purpose of their involvement in HTA, although some were able to cite specific examples of how their input made a difference; members of an HTA decision-making committee appeared to have a better understanding and were able to give examples. Feedback from HTA agencies to the patient groups on their submissions is often not provided.ConclusionsAlthough considerable progress has been made in terms of engaging patients and patient groups in HTA, gaps remain in how involvement is supported, including facilitating involvement, clarity on roles, two-way flow of information, and methods for enhancing communication between patient organisations and HTA agencies.

INTRODUCTION:In Bulgaria, the regulatory body sets for the first time legal requirements for Health Technology Assessment (HTA) in Law on Medicinal Products in Human Medicine (LMPHM) on 27 June 2015. The next essential step for HTA capacity building was the promulgation of Ordinance 9 / December 1, 2015 on the conditions and procedures for conducting health technology assessment by the Ministry of Health (1). In the beginning of 2016, the Main Price and Reimbursement Committee was set and launched a process for establishing the small working groups with the task of reviewing the first applicants reports of pharmaceuticals for inclusion in the Positive Drug List (PDL).METHODS:The objective of this study is to summarize the recommendations of the newly established HTA Committee in Bulgaria and to examine the characteristics of the technologies and the key considerations that led to those decisions. We systematically read all published by the Committee recommendations for 2016 and analyzed them under: type of recommendations (positive or negative for inclusion in PDL), population, specialization, type of service, type of justification and the impact on final conclusions.RESULTS:For the first year of its work the HTA Committee was able to assess fifteen technologies (pharmaceuticals) and only one received a negative recommendation (6 percent) from the working group. All the rest (n = 14; 94 percent) were recommended for funding. The final recommendation from the Main Price and Reimbursement Committee is available for four (27 percent) technologies – all positive for inclusion in PDL. All recommendations were connected with adults and in oncology (n = 4; 27 percent); heart diseases (n = 4; 27 percent); Chronic Obstructive Pulmonary Disease, COPD (n = 2; 13 percent); diabetes (n = 2; 13 percent); psoriasis (n = 2; 13 percent); Hepatitis C (n = 1; 7 percent). The only negative recommendation was justified due to lack of robust evidence, safety issues and credibility of HTA analysis (2).CONCLUSIONS:The information about the number of applications received from the Committee is not available and correct conclusions about the capability is not possible, but indirect circumstances, as the lack of well-trained HTA experts, certainly impede establishment of the small working groups and slow the assessment process (3). At this point it is clear that additional efforts are need to overcome the barriers and smooth adoption and implementation of HTA methods in Bulgaria.

While involving patients in health technology assessment (HTA) has become increasingly common and important around the world, little is known about the optimal methods of evaluating patients' involvement (PI) in HTA. This scoping review was undertaken to provide an overview of currently available methods for the evaluation of PI, specifically the impact of PI on HTA recommendations. A literature search was conducted using nine databases as well as a grey literature search of the websites of 26 organizations related to the conduct, practice or research of HTA to identify articles, reports and abstracts related to the evaluation of PI impact in HTA. We identified 1,248 unique citations, six of which met our eligibility criteria. These six records (five articles, and one report) were all published after 2012. Four assessed the impact of patient experience submissions on final HTA recommendations; one evaluated the impact of direct involvement on HTA committees, and one assessed impact of multiple forms of involvement. Methods of evaluation included quantitative analyses of reimbursement decisions, qualitative interviews with those directly involved in an assessment, surveys of patient groups and committee members, and the review of HTA reports. Quantitative evaluation of PI based on associations with funding decisions may not be feasible or fully capture the relevant impact of PI in the assessment of health technologies. Rather, a combination of both qualitative and quantitative strategies may allow for the most comprehensive assessment of the impact of PI on HTA recommendations when possible.

ObjectivesTo assess the health technology assessment (HTA) process in Greece from its establishment in 2018 until 2023 in terms of timeliness and productivity.MethodsData were collected from the HTA Committee’s database and other publicly available sources. The overall study timeframe was divided into three periods: (i) July 2018–January 2020, (ii) January 2020–July 2021, and (iii) July 2021–February 2023.ResultsDuring the study period, a total of 1,157 applications for medicinal products (MPs) (including 219 new active substances (NAS) and orphans) were submitted to the HTA Committee. The number of HTA recommendations increased from 60 (first period) to 641 (third period), while the backlog of MPs pending HTA and price negotiations decreased from 89 and 106 (January 2020) to 8 and 44 (February 2023), respectively. The median time intervals for all application types decreased significantly over time. In February 2023, the median time for clinical data assessment of NAS (excluding orphans) almost halved from 207 days in the first period to 114 days; median times for NAS and orphans from regulatory approval to HTA application were 420 and 457 days, and from HTA application to reimbursement 228 and 417 days, respectively.ConclusionsThe performance of the HTA process in Greece improved significantly over time, with increased MP appraisals, backlog reduction, and decreased timelines. Delays in reimbursement of NAS were mainly caused by the long gap between regulatory approval and HTA application. Overall, HTA review times in Greece are now on par with that of well-established European HTA systems.

Uncertainty tolerance among experts involved in drug reimbursement recommendations: Qualitative evidence from HTA committees in Canada and Poland

The aim of this study was to explore stakeholders' points of views regarding the applicability and relevance of a framework for user involvement in health technology assessment (HTA) at the local level. We tested this framework in the context of the assessment of alternative measures to restraint and seclusion among hospitalized adults and those living in long-term-care facilities. Twenty stakeholders (health managers, user representatives, and clinicians) from seven regions of Quebec participated in a semi-structured interview. A thematic analysis of the transcribed interviews was performed. The findings highlighted the relevance and applicability of the framework to this specific HTA. According to interviewees, direct participation of users in the HTA process allows them to be part of the decision-making process. User consultation makes it possible to consider the views of a wide variety of people, such as marginalized and vulnerable groups, who do not necessarily meet the requirements for participating in HTA committees. However, some user representatives emphasized that user consultation should be integrated into a more holistic and participatory perspective. The most frequent barrier associated with user involvement in HTA was the top-down health system, which takes little account of the user's perspective. The proposed framework was seen as a reference tool for making practitioners and health managers aware of the different mechanisms of user involvement in HTA and providing a structured way to classify and describe strategies. However, there is a need for more concrete instruments to guide practice and support decision making on specific strategies for user involvement in HTA at the local level.

BackgroundThere is a lack of focused research on the older population in Ghana and about issues pertaining to their access to healthcare services. Furthermore, information is lacking regarding the fairness in the access to these services. This study aimed to ascertain whether horizontal and vertical equity requirements were being met in the healthcare utilisation among older adults aged 50 years and above.MethodsThis study was based on a secondary cross-sectional data from the World Health Organization’s Study on global AGEing (SAGE) and adult health wave 1 conducted from 2007 to 2008 in Ghana. Data on 4304 older adults aged 50 years-plus were analysed. Bivariate and multivariable analyses were carried out to analyse the association between outpatient/inpatient utilisation and (1) socioeconomic status (SES), controlling for need variables (horizontal equity) and (2) need variables, controlling for SES (vertical equity). Odds ratios with 95% confidence intervals were calculated to analyse the association between relevant variables.ResultsHorizontal and vertical inequities were found in the utilisation of outpatient services. Inpatient healthcare utilisation was both horizontally and vertically equitable. Women were found to be more likely to use outpatient services than men but had reduced odds of using inpatient services. Possessing a health insurance was also significantly associated with the use of both inpatient and outpatient services.ConclusionWhilst equity exists in inpatient care utilisation, more needs to be done to achieve equity in the access to outpatient services. The study reaffirms the need to evaluate both the horizontal and vertical dimensions in the assessment of equity in healthcare access. It provides the basis for further research in bridging the healthcare access inequity gap among older adults in Ghana.

IntroductionInvolving patients in the health technology assessment (HTA) lifecycle is a core principle at the National Institute for Health and Care Excellence (NICE). We include both patient organizations and patient experts, which helps build public confidence in healthcare decision-making. We continually work with patient experts to improve their experience and ability to participate by seeking patient expert feedback after every committee meeting.MethodsWe sent patient experts an anonymous experience survey containing a five-point Likert scale and open text boxes to capture qualitative data. The survey covered their overall experience, interaction with the committee Chair, and the support they received from both NICE and the Public Involvement Programme (PIP).In the 2019 to 2020 period we sent out 59 questionnaires and received 29 responses (47%), all of which were from medicines HTA committee participants. In the 2020 to 2021 period we sent out 120 questionnaires and received 65 responses (54%), of which 64 were from patient experts who attended medicines HTAs and one was from a medical devices HTA committee participant.ResultsGood or excellent experiences were reported by 90 percent of patient experts. The four main success factors noted were: good support before meetings; being welcomed and respected; well organized meetings; and patient expert input being valued. Areas for further improvement included: providing better briefing before meetings; allowing more time to review documents; providing more technical support; and giving more consideration to the opinions of patient experts.ConclusionsAs a result of the feedback received, the PIP now holds monthly group briefing meetings for patient experts. We also publish the anonymized feedback from the patient experts quarterly in a newsletter for committee members and share the data with internal NICE teams. Additionally, NICE aims to: send committee papers out earlier; have the option of holding a technical engagement call before committee meetings; and develop a feedback mechanism to ascertain the impact of patient input.

A large body of research has demonstrated that child benefit systems are of paramount importance in reducing child poverty, thus having an important vertical equity component. Although all child benefit systems embody in one way or the other such vertical equity objective, the primary objective of child benefit systems is to (at least partly) compensate for the costs associated with childrearing and to minimize the welfare loss relative to childless families, a horizontal equity objective. Most studies are concerned with vertical equity and child poverty reduction; here we also explicitly take the dimension of horizontal equity into account. In this paper, we propose and develop a two-dimensional framework for evaluating and classifying the outcomes of child benefit systems in terms of both vertical and horizontal equity. Treating these two objectives as analytically distinct permits the construction of a synthetic index of child benefit outcomes and allows for the explicit incorporation of a value judgement about the most important objective of child benefit systems. In doing so, we propose a novel measure for gauging horizontal equity based on the cost of children implicit in commonly used equivalence scales drawing on the public finance literature. We demonstrate the potential of our evaluative framework for policy purposes by means of an empirical application for 31 European welfare states. We contribute to the literature by highlighting the role of characteristics of benefit systems in achieving certain objectives regarding horizontal and/or vertical equity.

IntroductionInvolving patients in the health technology assessment (HTA) lifecycle is a core principle at the National Institute for Health and Care Excellence (NICE) that helps build public confidence in healthcare decision-making. The last stage in this lifecycle is an appeal whereby stakeholders, including patient organizations, can appeal against the HTA committee’s decision based on two specific grounds: (i) the committee has failed to act fairly or NICE has exceeded its powers; and (ii) the recommendation is unreasonable considering the evidence submitted to NICE. Improving patient involvement in the appeals process was identified as a key area from consultative work undertaken with patient organizations in 2019.MethodsIn September 2020, surveys were sent to patient organizations for their feedback. The organizations received a survey tailored to their circumstances in relation to the following four outcomes.(i)An appeal was lodged that: (a) resulted in an appeal hearing; or (b) did not result in an appeal hearing.(ii)An appeal was not lodged but the organization had received: (a) negative guidance; or (b) ‘optimized’ guidance.ResultsSixteen responses were received across the four surveys. The key findings in priority order were as follows.(i)The legalist nature of appeal hearings is off-putting.(ii)The online appeals guide and template letter are not easy to use.(iii)Organizations would like more information on appeal hearings and timelines.(iv)Organizations would like patient-friendly materials and training on par with the support they receive for other HTA participation.ConclusionsAs a result of these findings, we are iteratively developing materials to support patient organizations, including:•a glossary;•a redesigned appeals web page with clearer timelines and a template letter;•a ‘Frequently Asked Questions’ document specifically for patient organizations; and•a video podcast from a lay appeals panel Chair explaining appeals in a patient-friendly way.We are also including a section on appeals in our introductory training for all patient organizations participating in medicines HTAs.

This paper addresses some of the issues related to health care in the United States. In so doing, I give primary emphasis to questions involving access to health care. Even so, I limit the discussion of any particular topic to its most important facets. The principle of selection involves various criteria: insofar as possible, I discuss those issues that are important at a system level (particularly as they impinge on the allocation of resources), that involve economic arrangements influencing behavior and performance, and that can be illuminated by the economist's perspective. I attempt to give primary emphasis to those variables whose influence is far ranging. In so doing, of course, we cannot examine every network of interrelations. Although this simplifies the discussion, we pay a price for incompleteness. There is little choice, however. One of the difficulties with the health field stems from the fact that everything is interrelated: that intervention on one front has “side effects” on other fronts; that intervention designed to accomplish one purpose sometimes fails to do so because other factors that appear unrelated are not changed. To discuss everything is impossible. On the other hand, to say nothing because we cannot discuss everything is irresponsible. Finally, we will have to reach judgments even in spite of the relative weakness of the data available for analysis. We know far less, for example, about the availability of services (particularly if corrected for quality differentials) than we would like to. So, too, with the impact of differences in utilization on levels of health. The current refrain often seems to be, “But we have no output measures.” That, regrettably, is frequently the case. Nevertheless, just as we cannot be silent because of the complexity of intertwined relations, neither can we be silent because of insufficiency of data. We can use experience and judgment to arrive at (tentative) conclusions. Not knowing everything does not mean we know nothing.