Health Technology Advisory Committee Research Articles

Evaluating multi-criteria decision-making in health technology assessment

ObjectivesLiterature has emphasized the role of multi-criteria decision analysis (MCDA) in improving consistency and comprehensiveness of health technology assessment (HTA), however there has been no evidence to indicate this as true. We sought to understand if MCDA ensures systematic and comprehensive HTA. MethodsIn Canada, the Ontario Health Technology Advisory Committee (OHTAC) was created as a provincial portal for funding recommendations regarding introduction of nondrug health technologies (HTs). In 2009, a “Decision Determinants Sub-Committee” developed a MCDA framework outlining specific criteria to be considered. We undertook mixed-methods analyses to investigate funding recommendation reports, including types of evidence used to evaluate criteria, strength and quality of evidence, and the relationship between criteria and funding recommendations. ResultsThe findings indicated that comprehensive and consistent use of evidence is lacking even with MCDA. Effectiveness was almost always considered, whereas ethical values were considered in only one-third of reports. Evaluation of quality and strength of evidence declined over the years, and value for money was often not assessed. ConclusionsWe have proposed two reasons for the inconsistent evaluation: (1) insufficient planning for evaluation (i.e., what needs to be evaluated), and (2) lack of evidence. Further research could focus on embedding economic and societal/ethical considerations in the MCDA framework, and specifying a system to evaluate the efficacy of these variables. Such an approach may encourage comprehensive and consistent consideration of diverse types of evidence in HTA.

DEVELOPMENT OF THE ONTARIO DECISION FRAMEWORK: A VALUES BASED FRAMEWORK FOR HEALTH TECHNOLOGY ASSESSMENT.

In 2007, the Ontario Health Technology Advisory Committee (OHTAC) developed a decision framework to guide decision making around nondrug health technologies. In 2012, OHTAC commissioned a revision of this framework to enhance its usability and deepen its conceptual and theoretical foundations. The committee overseeing this work used several methods: (a) a priori consensus on guiding principles, (b) a scoping review of decision attributes and processes used globally in health technology assessment (HTA), (c) presentations by methods experts and members of review committees, and (d) committee deliberations over a period of 3 years. The committee adopted a multi-criteria decision-making approach, but rejected the formal use of multi-criteria decision analysis. Three broad categories of attributes were identified: (I) context criteria attributes included factors such as stakeholders, adoption pressures from neighboring jurisdictions, and potential conflicts of interest; (II) primary appraisal criteria attributes included (i) benefits and harms, (ii) economics, and (iii) patient-centered care; (III) feasibility criteria attributes included budget impact and organizational feasibility. The revised Ontario Decision Framework is similar in some respects to frameworks used in HTA worldwide. Its distinctive characteristics are that: it is based on an explicit set of social values; HTA paradigms (evidence based medicine, economics, and bioethics/social science) are used to aggregate decision attributes; and that it is rooted in a theoretical framework of optimal decision making, rather than one related to broad social goals, such as health or welfare maximization.

Insurance coverage issues associated with bariatric surgery in the US

Obesity has become a global epidemic. In the UK, the rate of obesity has risen by about 1% per year since the mid-1990s, and in 2009 about 25% of UK adults were obese and 57% were overweight. The US is also experiencing this epidemic. While new data continue to emerge in support of bariatric surgery as a way of combating the obesity epidemic, it is not a miracle solution. Despite its proven cost-effectiveness, the public funding of bariatric surgery is frequently questioned based on ethical arguments relating to the self-inflicted or non-disease nature of obesity. The aim of this paper is to explore issues surrounding the provision of insurance coverage for bariatric surgery in the US, using four main criteria from the Ontario Health Technology Advisory Committee 2010 report, to weigh the decision determinants for payment of bariatric surgery.

Reimbursement programs and health technology assessment for diabetes devices and supplies: a Canadian perspective.

The reimbursement model for medical devices and supplies used in the management of diabetes in Canada, along with the process for assessing new health technologies, can be complicated. Various provincial programs, including Ontario's Assistive Devices Program and the Ontario Monitoring for Health Program, reimburse the costs associated with certain devices and supplies for diabetes management. In addition, provincial advisory committees, such as the Ontario Health Technology Advisory Committee, review and make recommendations on the adoption of new health technologies in each province. This article provides an overview of the reimbursement programs available for diabetes devices and supplies and reviews the process for assessing new health technologies using the province of Ontario as an example.

Puzzling about problems: the ambiguous search for an evidence-based strategy for handling influx of health technology

This paper focuses on problem frame differences among actors (members of an advisory body, senior administrators and clinical unit managers) who are concerned with the introduction of new health technology at the regional level in Sweden. It explores issues related to problem framing, puzzling, powering, participation and the various rationales articulated in the ambiguous search for an evidence-based strategy to handle the influx of new technologies. The Health Technology Advisory Committee (HTAC) was established in one Swedish county council in 2004 with the intention of controlling both the introduction of health technology and supporting policy decision and clinical practice by promoting the use of best evidence. The HTAC followed a scientific rationality dominated by one problem frame, although the problematic situation, as it was framed by all the actor groups, was highly complex and not solely a matter of evidence. This paper illustrates how problem frame differences shape the puzzling of a policy problem and how the different distinguishable policy styles are dependent on who is participating and who is not participating in the puzzling.

ASSESSING THE IMPACTS OF CITIZEN DELIBERATIONS ON THE HEALTH TECHNOLOGY PROCESS

We assessed the impacts of a Citizens' Reference Panel on the deliberations of a provincial health technology advisory committee and its secretariat, which produce, recommendations for the use of health technologies in Ontario, Canada. A fourteen-member citizens' reference panel was convened five times between February 2009 and May 2010 to participate in informed, facilitated discussions to inform the assessment of individual technologies and provincial health technology assessment processes more generally. Qualitative data collection methods were used to document observed and perceived impacts of the citizens' panel on the health technology assessment (HTA) process. Panel impacts were observed for all technologies reviewed, at two different stages in the HTA process, and represented macro- (raising awareness) and micro-level (informing recommendations) impacts. These impacts were shaped by periodic opportunities for direct and brokered exchange between the Panel and the expert advisory committee to clarify roles, foster accountability, and build trust. Our findings offer new insights about one of the main considerations in the design of deliberative participatory structures-how to maintain the independence of a citizens' panel while ensuring that their input is considered at key junctures in the HTA process. Citizens' panels can exert various impacts on the HTA process. Ensuring these types of structures include opportunities for direct exchange between citizens and experts, to clarify roles, promote accountability, and build trust will facilitate their impacts in a variety of settings.

Eliciting ethical and social values in health technology assessment: A participatory approach

Despite a growing consensus that ethical and social values should be addressed in health technology assessment (HTA) processes, there exist a variety of methods for doing so. There is growing interest in involving citizens in policy development to ensure that decisions are legitimate, and reflect the broad social values of the public. We sought to bring these issues together by employing a participatory approach to elicit ethical and social values in HTA. Our primary objective was to elicit a set of ethical and social values from citizens that could be used to guide Ontario’s HTA evidentiary review and appraisal process. A secondary objective was to explore the feasibility of using participatory approaches to elicit these values.A 14-person Citizens’ Reference Panel on Health Technologies was established to provide input to the Ontario Health Technology Advisory Committee in developing its recommendations. A mixed methods approach was used where informed, deliberative discussions were combined with pre- and post-questionnaires, which assessed the relative importance of various ethical and social values as well as their stability over time.Over the course of five meetings, panel members progressed toward the identification of a set of core values –universal access, choice and quality care. These values were consistently prioritized as the core values that should be considered in the evaluation of health technologies and ensuing recommendations. Sustained and deliberative methods, like a citizens’ panel, offer a promising approach for eliciting ethical and social values into HTA.

Conditionally Funded Field Evaluations and Practical Trial Design Within a Health Technology Assessment Framework

Randomized controlled trials may lack generalizability and cannot always provide sufficient information for policymakers to make decisions regarding the use of new technologies entering into a health care system. Information related to other aspects of health technologies, in addition to clinical endpoints, such as economic, patient-related, and organizational issues, may be needed before making policy decisions. To address generalizability issues related to a health care technology, primary data collection conducted within a health care system may be required. A process for primary data collection to inform health policy in Ontario regarding health care technologies was initiated in 2003. Following recommendations from the Ontario Health Technology Advisory Committee, conditionally funded field evaluations for promising technologies with insufficient information are initiated. The conduct of these field evaluations occurs in 3 stages. The translation of the policy problem to research questions that can be answered through the design of a practical clinical trial is the first stage, followed by implementation, data collection, quality control, and quality assurance activities. After the completion of the field evaluation data collection and analysis, the results of the study are presented to the committee to respond to its request. Technologies evaluated by this process include drug-eluting stents, the endovascular repair of abdominal aortic aneurysms, and multifaceted interventions in diabetes care. This process, developed in Ontario for the evaluation of health technologies, has been a successful collaborative effort between practitioners, academia, government, health policymakers, and industry that has permitted an evidence-based framework for the approval and funding of technologies within the health care system.

Health technology assessment: A comprehensive framework for evidence-based recommendations in Ontario

This study describes the development of a framework for health technology decisions, for Ontario Health Technology Advisory Committee (OHTAC) in Ontario, Canada. OHTAC convened a "Decision Determinants Sub-Committee" in January 2007, which undertook a systematic literature review and conducted key informant interviews to develop an explicit decision-making framework. The "Decision Determinants Sub-Committee" offered recommendations about decision criteria, and the process by which decisions are made. Decision criteria include (i) overall clinical benefit, (ii) consistency with societal and ethical values, (iii) value for money, and (iv) feasibility of adoption into the health system. The decision process should be transparent and fair and should use a deliberative process in delivering recommendations. This methodology is currently being pilot tested in a live environment: OHTAC. It will be evaluated and revised according to its feasibility, acceptability, and perceived usefulness.

Vascular Ultrasound Screening for Asymptomatic Abdominal Aortic Aneurysm

This health technology assessment examines vascular ultrasound screening for abdominal aortic aneurysm (AAA) in asymptomatic populations. Screening reduces the incidence of AAA ruptures, rates of emergency surgical repair and AAA-attributable mortality in males ages 65 to 74. The benefit of screening women has not been established. Ontario data suggest that AAA is underdiagnosed in women, and that women are systematically undertreated. Targeting smokers for screening was found to maximize cost-effectiveness. Economic analysis found that screening may generate savings from the avoidance of emergency surgeries. Based on these findings, the Ontario Health Technology Advisory Committee has recommended screening for AAA in both male and female ever-smokers ages 65 to 74.

Establishing a comprehensive continuum from an evidentiary base to policy development for health technologies: The Ontario experience

The aim of this study was to describe a comprehensive continuum that has developed in Ontario between government and key stakeholder groups, including hospitals, physicians, academic institutions, clinical epidemiologists, health economists, industry, and bioethicists to achieve evidence-based recommendations for policy development. The various components of the comprehensive model that has evolved to develop an evidentiary platform for policy development are summarized, and the flow between these components is described. The development of the Ontario Health Technology Advisory Committee (OHTAC) and associated programs demonstrate the need to go beyond the traditional steps taken within most health technology assessment paradigms. These components include pragmatic postmarketing studies, human factors, and safety analyses, and formalized interactions with a broad spectrum of potential end-users of each technology, experts, and industry. Thesecomponents, taken together with an expanded systematic review to include a range of economic analyses, and societal impacts augment the traditional systematic review processes. This approach has been found to be important in assisting decision making and has resulted in an 81 percent conversion from evidence to policy consideration for eighty-three technologies that had been assessed at the time this article was submitted. The comprehensive model, centered around OHTAC, has added important new dimensions to health policy by improving its relevance to decision makers and providing an accountable and transparent basis for government to invest appropriately in health technologies. This study could also form a basis for further research into appropriate methodologies and outcome measurements as they relate to each component of this approach.

Building Bridges Between Academic Research and Policy Formulation

Healthcare expenditure in Ontario has been escalating at increasing rates and a growing proportion of this has come from medical devices and other technologies. A comprehensive evidence-based process for the assessment of new health technologies in Ontario was recently established (the Health Technology Policy Analysis [HTPA] process). Requests for funding for new technologies are directed through a department of the Ministry called the Medical Advisory Secretariat (MAS) who, using a systematic and evidence-based approach, provides information on the safety, effectiveness, and cost effectiveness of the technology to the Ontario Health Technology Advisory Committee (OHTAC). If the OHTAC determines that the evidentiary base for making recommendations is too uncertain, a request is made to reduce this uncertainty through the conduct of a 'real world' Ontario-based study. The Program for Assessment of Technology in Health (PATH) reduction of Uncertainty through Field Evaluation (PRUFE) framework is an iterative process for collecting new information, updating prior knowledge and providing new 'real world' evidence back to the Ministry regarding the cost effectiveness of new health technologies. MAS/OHTAC has undertaken evidence-based analyses for >50 new technologies. The PRUFE framework has been successful for reducing uncertainty and controlling healthcare expenditures while providing access and improving patient outcomes.

In Search of a Technical Fee for Endoscopy in Ontario

In Search of a Technical Fee for Endoscopy in Ontario

Health technology assessment, research, and implementation within a health region in Alberta, Canada.

To determine the need for and implement health technology assessment (HTA) to inform decision making and policy within a regional health care system in Calgary (Alberta, Canada). Published literature and organizational materials for the Calgary Health Region (CHR) and HTA units worldwide were reviewed. Key individuals within the provincial health ministry (Alberta Health and Wellness), CHR, the University of Calgary (U of C), funding agencies, and HTA organizations were consulted in a structured fashion. A structure for a regional HTA program was developed, taking into account relationships between these organizations. A locally focused HTA and implementation unit was deemed desirable. The Calgary Health Technology Implementation Unit (CaHTIU) was established. The CaHTIU was designed to efficiently integrate with CHR planning as well as undertake independent research activities. HTA activities focus primarily on CHR needs and are managed by a Health Technology Advisory Committee (HTAC) that consists of CHR management and other key individuals. Working groups contribute to and coordinate HTAs and implementation under the leadership of the unit Director, and include content as well as management individuals. The unit cooperates where appropriate with extant Canadian HTA organizations. The Calgary HTA unit is unique in Canada, because it functions within a regional health care system as well as a research institution. Advantages include a local focus in terms of applied HTAs, a systematic process for implementation of recommendations, and a collaborative atmosphere for research within the U of C.

The evolution of heart, lung and liver transplantation services in Australia

The introduction and operation of heart, lung and liver transplantation services in Australia have been assisted by a Nationally Funded Centres policy adopted by health ministers. Assessment for eligibility for support under the program has been undertaken by the Australian Health Technology Advisory Committee. Outcomes/survival rates at transplantation centres have compared favourably with results from other countries. As in other countries, shortage of donor organs has limited the number of procedures undertaken. Some concern remains regarding the viability of newer transplantation programs which are now being established in smaller centres of population.

Health care technology in Australia

The health care system in Australia is pluralistic, complex, and only loosely organized. The Commonwealth government is primarily concerned with funding programs and the development of broad policies. The introduction and diffusion of health care technologies in Australia is determined by a complex interaction of market forces, public funding, and regulation. Australia became involved in health care technology assessment in 1982. In 1990, activities were reorganized and the Australian Health Technology Advisory Committee (AHTAC) was formed at the national level. Despite limited funding, Australia has had some significant successes in informing policy through appropriately targeted, well-timed assessment.