Just the Job

Multicriteria Decision Analysis to Support Health Technology Assessment Agencies: Benefits, Limitations, and the Way Forward

.In moving toward malaria elimination, finer scale malaria risk maps are required to identify hotspots for implementing surveillance–response activities, allocating resources, and preparing health facilities based on the needs and necessities at each specific area. This study aimed to demonstrate the use of multi-criteria decision analysis (MCDA) in conjunction with geographic information systems (GISs) to create a spatial model and risk maps by integrating satellite remote-sensing and malaria surveillance data from 18 counties of Yunnan Province along the China–Myanmar border. The MCDA composite and annual models and risk maps were created from the consensus among the experts who have been working and know situations in the study areas. The experts identified and provided relative factor weights for nine socioeconomic and disease ecology factors as a weighted linear combination model of the following: ([Forest coverage × 0.041] + [Cropland × 0.086] + [Water body × 0.175] + [Elevation × 0.297] + [Human population density × 0.043] + [Imported case × 0.258] + [Distance to road × 0.030] + [Distance to health facility × 0.033] + [Urbanization × 0.036]). The expert-based model had a good prediction capacity with a high area under curve. The study has demonstrated the novel integrated use of spatial MCDA which combines multiple environmental factors in estimating disease risk by using decision rules derived from existing knowledge or hypothesized understanding of the risk factors via diverse quantitative and qualitative criteria using both data-driven and qualitative indicators from the experts. The model and fine MCDA risk map developed in this study could assist in focusing the elimination efforts in the specifically identified locations with high risks.

BACKGROUND: In the context of the progressive movement towards patientcentred care, patient-specific decision support is an important focus of interest. Many diagnostic and treatment patient decision aids (PDAs) are now available to help patients make informed choice decisions. An increasing number of these are software-based, with some available online. Multi-Criteria Decision Analysis (MCDA) is a potentially useful technique on which to base a software-assisted PDA, especially when the decision is complex - as is the case in choosing the best treatment for non-small cell lung cancer – but it has so far been relatively little exploited in this area. The use of any from a number of existing MCDA-based software applications in the development and delivery of a MCDA-based interactive PDA can be an effective way of achieving “best-practice” or normative standards of decision making, such as 1) a well-constructed set of decision criteria or 2) logically consistent patient preferences. However, it also involves the use of resources such as the time and cognitive effort involved in decision-making. The comparative evaluation of alternative MCDA-based software applications in developing and delivering a PDA therefore involves trade-offs between decision effectiveness and decision resource criteria moving from the normative to the prescriptive. MCDA is an ideal tool for this meta-evaluation task as well as for the adoption decision itself. AIM: To analyse, as proof of concept, the use of MCDA for the development, implementation and evaluation of interactive PDAs in routine clinical practice. OBJECTIVES: 1. To assess the use with clinicians in the Spanish NHS of two alternative MCDA software applications which implement dissimilar MCDA techniques in the development of a PDA in routine clinical practice; 2. To assess the use with clinicians in the Spanish NHS of the same two alternative MCDA software applications in the implementation of a PDA in an environment replicating actual clinical consultations; 3. To build a meta-multi-criteria decision model based on the Decision Resources Decision Effectiveness Analysis (DRDEA) framework and assess the use of this model by clinicians in the Spanish NHS to make the choice between the two MCDA applications as the basis for a PDA. METHODS: 1) Two dissimilar MCDA software applications served as a basis for the development of a lung cancer clinical management PDA in close collaboration with two different groups of three clinicians from two different Spanish NHS hospitals (H1 and H2): 1) Expert Choice, which implements the Analytic Hierarchy Process (AHP) MCDA approach; 2) Annalisa in Elicia (ALEL), which implements the Simple Attribute Weighting (SAW) MCDA approach. The process of codevelopment of the PDA in hospitals H1 and H2 was documented; 2) Expert Choice was used to implement (i.e. deliver) the lung cancer clinical management PDA in three hypothetical consultations in hospital H1. In each consultation, one of the three clinicians involved in the development of the tool, with support by this researcher, guided a proxy patient (a non-clinical member of hospital staff) through the PDA. The same process was repeated with the MCDA software ALEL in hospital H2. The process of delivery of the PDA in hospitals H1 and H2 was documented; 3) This researcher built a meta-multi-criteria decision model based on the DRDEA framework to help clinicians choose between different MCDA software applications as the basis of a PDA. The MCDA approach used for this meta-model was Multi- Attribute Value Theory (MAVT). The model was implemented, using the software HiView 3, with three clinicians from hospital H3 for the choice between Expert Choice and ALEL as the basis of a lung cancer clinical management PDA. RESULTS: The thesis makes a three-fold contribution to research in patient-centred decision support. First, it presents two new MCDA software-based approaches to clinical decision support, based on joint work with clinicians in the Spanish NHS, for developing an interactive PDA for the clinical management of non-small cell lung cancer. Second, it describes the use of these decision support tools in the delivery of 5 an interactive PDA for the clinical management of non-small cell lung cancer in a hospital environment via simulated consultations between actual clinicians, with support from this researcher, and proxy lung cancer patients. Third, it presents and applies a new MCDA-based methodology for evaluating the use of alternative MCDA software applications in the development and delivery of interactive PDAs.

<p>THEORETICAL FOUNDATIONS AND USE OF MULTI-CRITERIA DECISION ANALYSIS IN THE PHARMACEUTICAL SECTOR OF UKRAINE ACCORDING TO THE EUROPEAN REFORMING VECTOR</p><p>O.B. Piniazhko, O.M. Zaliska</p><p>Danylo Halytsky Lviv National Medical University</p><p> </p><p>The reforming of health care system has been started to carry out in Ukraine. Assessing the value of medical technologies may require new approaches that take into account a more comprehensive set of parameters than the incremental cost-effectiveness ratio, ICER/ quality adjusted life years, QALY.</p><p>Technological advancements in combination with higher life expectancy, higher patient expectations, and increased prevalence of chronic diseases, have led to significant increases in public spending on pharmaceuticals, which, on average account for 9,3% of total health expenditure or about 1,4% of GDP across OECD countries (OECD, 2012). Given the resources governments and health systems can spend on healthcare, the pathway to optimal resource allocation passes through cost containment and efficiency improvement policies. </p><p>In the context of multi-criteria decision analysis (MCDA), elements of value can be measured and scored in their natural units or through constructed scales, quantitatively or qualitatively, and weights are assigned to reflect criteria’s importance when combining them. This approach may provide a more comprehensive account of value parameters, greater transparency in how multiple criteria are explicitly valued, weighted, and aggregated, and a more inclusive approach to stakeholder views on value.</p><p>Our aim was to perform a comparative analysis of the main approaches, methods and stages of MCDA, to determine the urgent and relevant use of MCDA in health care system of Ukraine based on the experience of developed countries.</p><p>We have analyzed the ukrainian and foreign publications, articles, systematic reviews, practical guidelines on MCDA in health care using PubMed, EMBASE, Internet resources as search tools.</p><p>Decision analysis can provide an alternative way of measuring and eliciting value. In particular, MCDA is both an approach and a set of techniques, with the goal of providing an overall ordering of options by looking at the extent to which a set of objectives are achieved. MCDA methodologies have been suggested for use in public services since 1960. MCDA has been used, on an experimental basis, in order to assess the benefit-risk (clinical) profile of new medicines for the purpose of regulatory approval during marketing authorisation stage by the European Medicines Agency and others. Its use has also been suggested in health care and value assessment in HTA also offering a conceptual framework.</p><p>Although a variety of MCDA methodologies exist, the process of MCDA includes a number of common stages as follows: 1) establishing the decision context by defining the aims of the MDCA, and who the decision makers and other key stakeholders are; 2) identifying the objectives and criteria that reflect the value associated with the consequences of each option; 3) «scoring» the value associated with the performance of each option against the criteria; 4) «weighting» each of the criteria to reflect their relative importance to the decision; 5) examine the results; 6) conducting a sensitivity analysis of the results to test the influence of changes in scores or weights.</p><p>As the results of our review we suggested the urgent and relevant use of MCDA in health care system of Ukraine based on the experience of developed countries. The comparative analysis of the main approaches, methods and stages of MCDA was performed and trends of it’s implementation in the pharmaceutical industry of our country were suggested.</p><p>Due to the performed analysis, in our opinion in order to optimize and improve the decision-making process in health care the implementation of MCDA approach will provide transparent and consistent management decisions in Ukraine.</p>

The Use of MCDA in HTA: Great Potential, but More Effort Needed

Qualitative methods tend to be used to incorporate patient preferences into healthcare decision making. However, for patient preferences to be given adequate consideration by decision makers they need to be quantified. Multi-criteria decision analysis (MCDA) is one way to quantify and capture the patient voice. The objective of this review was to report on existing MCDAs involving patients to support the future use of MCDA to capture the patient voice. MEDLINE and EMBASE were searched in June 2014 for English-language papers with no date restriction. The following search terms were used: 'multi-criteria decision*', 'multiple criteria decision*', 'MCDA', 'benefit risk assessment*', 'risk benefit assessment*', 'multicriteri* decision*', 'MCDM', 'multi-criteri* decision*'. Abstracts were included if they reported the application of MCDA to assess healthcare interventions where patients were the source of weights. Abstracts were excluded if they did not apply MCDA, such as discussions of how MCDA could be used; or did not evaluate healthcare interventions, such as MCDAs to assess the level of health need in a locality. Data were extracted on weighting method, variation in patient and expert preferences, and discussion on different weighting techniques. The review identified ten English-language studies that reported an MCDA to assess healthcare interventions and involved patients as a source of weights. These studies reported 12 applications of MCDA. Different methods of preference elicitation were employed: direct weighting in workshops; discrete choice experiment surveys; and the analytical hierarchy process using both workshops and surveys. There was significant heterogeneity in patient responses and differences between patients, who put greater weight on disease characteristics and treatment convenience, and experts, who put more weight on efficacy. The studies highlighted cognitive challenges associated with some weighting methods, though patients' views on their ability to undertake weighting tasks was positive. This review identified several recent examples of MCDA used to elicit patient preferences, which support the feasibility of using MCDA to capture the patient voice. Challenges identified included, how best to reflect the heterogeneity of patient preferences in decision making and how to manage the cognitive burden associated with some MCDA tasks.

Recent guidelines for risk-informed decision making (RIDM) provide a gold-standard for how to incorporate probabilistic risk models in conjunction with other considerations in a decision process. Nevertheless, risk quantification using probabilistic and statistical methods is difficult in situations where threat, vulnerability, and consequences are highly uncertain and risk quantification. In such situations a wider variety of methods could be employed, which we call decision making informed by risk (DMIR) combining risk and decision analytics. Risk informed decision making (RIDM) can be considered as a special case of DMIR. Multi criteria decision analysis (MCDA) often serves as a basis for DMIR in order to flexibly accommodate different levels of analytical detail. DMIR often involves artful use of proxy variables that correlate with, and are more measurable than, underlying factors of interest. This article introduces the notion of DMIR and discusses the use of MCDA in its application in the context of risk-based problems. MCDA-based risk analyses identify metrics associated with threats of concern and system vulnerabilities, characterize the way in which alternative actions can affect these threats and vulnerabilities, and ultimately synthesize this information to compare, prioritize, or select alternative mitigation strategies. Simple linear additive MCDA models often integrate these inputs, but the same simplicity can limit such approaches and create pitfalls and more advanced models including multiplicative relationships can be warranted. This essay qualitatively explores the critical practitioner questions of how and when the use of linear multicriteria models creates significant problems, and how to avoid them.

Weight of evidence (WOE) methods are key components of ecological and human health risk assessments. Most WOE applications rely on the qualitative integration of diverse lines of evidence (LOE) representing impact on ecological receptors and humans. Recent calls for transparency in assessments and justifiability of management decisions are pushing the community to consider quantitative methods for integrated risk assessment and management. This article compares and contrasts the type of information required for application of individual WOE techniques and the outcomes that they provide in ecological risk assessment and proposes a multicriteria decision analysis (MCDA) framework for integrating individual LOE in support of management decisions. The use of quantitative WOE techniques is illustrated for a hypothetical but realistic case study of selecting remedial alternatives at a contaminated aquatic site. Use of formal MCDA does not necessarily eliminate biases and judgment calls necessary for selecting remedial alternatives, but allows for transparent evaluation and fusion of individual LOE. It also provides justifiable methods for selecting remedial alternatives consistent with stakeholder and decision-maker values.

Groundwater seepage into open pit mines must be controlled carefully. Slope instability, dewatering of blast holes, and mining operations below the groundwater table are important issues caused by groundwater seepage into the Gol-Gohar open pit mine, Iran. There are several methods to overcome these problems, such as construction of cut-off walls, ditches and sumps, horizontal drains, and pumping wells. Drilling of a new pumping well has several difficulties in which the determination of its location is a major issue. In this study, a stochastic simulation approach called simulated annealing was used to determine the best possible locations for new pumping wells. Three major groundwater variables, including the groundwater level, electrical conductivity, and transmissivity were selected for the geostatistical study. The results of simulations showed reliable correlation (Pearson) between the variables. Comparison of the variograms at different depths of the Gol-Gohar pit mine revealed that the effect of faults intensified with increasing depth. The best potential locations for drilling of new pumping wells were identified by the use of multi-criteria decision analysis performed on the simulation results. This method can be used in other regions with similar hydrogeological settings.

Introduction:The use of multi-criteria decision analysis (MCDA) in health technology assessment (HTA) studies has become more common due to the fact that MCDA offers a comprehensive technique for decisions that involve multiple criteria and stakeholders. How MCDA contributes to the HTA decision making process is an issue to be investigated. A systematic review was carried out in order to provide an overview of the benefits identified in MCDA applications for the strategic HTA decision making process.Methods:A systematic review developed by Philip Wahlster et al. (2014) was updated. The papers were analyzed in order to determine how MCDA is connected with traditional HTA, and to identify opportunities through the application of MCDA. In total 965 papers were found, and 43 articles were included in the review. The included articles detailed MCDA applications oriented to tactical and strategic decision making processes. The review was conducted by two researchers.Results:Of the available studies published on MCDA, 76 percent were published between 2014 and 2017, and 24 percent were published prior to 2014. Regarding the MCDA methodology defined in the included studies, 10 used the analytical hierarchy process, four used multi-attribute theory, and others refer the methodology only as “MCDA”. Seventeen studies also included health technology economic analysis, in special cost-effectiveness, safety and technological innovation. The studies suggest MCDA adds value since it allows different stakeholders to be engaged in the decision making process.Conclusions:The increase in studies on MCDA and healthcare point to the possibility to add different criteria, engage people with different knowledge levels, and make the decision-making process more transparent. In comparison with other technical areas, the use of MCDA in healthcare is more focused on achieving the decision about adding the new technology, and to show how to engage stakeholders than to explain how to develop the algorithms and methodologies.

PHP134 - Parameter Uncertainty in Value Based Multi Criteria Decision Analysis: A Systematic Review of Methods

ABSTRACTNuclear accidents do not occur frequently, but their biological, psychosocial, and/or economic consequences may be severe. Hence, a thorough preparation for nuclear emergencies is needed to provide appropriate actions. During the transition phase of an accident, it is vital to include stakeholders in the decision‐making process in order to gain support for the recovery strategy to be implemented as well as to share different perspectives, knowledge, and views on the decision problem. Because nuclear accidents are complex, involving many relevant factors that range from technical aspects such as health effects and costs to nontechnical issues such as social acceptance, a multicriteria decision analysis (MCDA) may facilitate the decision‐making process. The aim of this study was to investigate the usefulness of MCDA in the transition phase of a nuclear accident. To this end, an MCDA tool, which uses the weighted sum of a set of normalized criteria, was explored in exercises carried out in panel meetings with a selected set of (largely) governmental stakeholders. The panel meetings were performed in the Netherlands and the Slovak Republic. The exercises were based on a fictitious case study that affected the urban environment of a small city. Prior to the meetings, a set of 8 possible recovery strategies was identified. The use of the MCDA tool showed that it facilitated the decision‐making process because it allowed for a structured and transparent approach in which stakeholders with diverse backgrounds can express their opinions and perspectives and reach consensus on the most appropriate recovery strategy. As such, it could be applied to a broader field of research involving any chemical release that necessitates an extended recovery strategy. Future research is needed in order to incorporate psychosocial effects of a nuclear accident as well as a broader group of stakeholders in exercises. Integr Environ Assess Manag 2021;17:376–387. © 2020 The Authors. Integrated Environmental Assessment and Management published by Wiley Periodicals LLC on behalf of Society of Environmental Toxicology & Chemistry (SETAC)

The mismatch between the health needs of populations affected by emergencies and resources devoted to response is projected to further increase. Making the response more effective is one of the solutions to meet the growing needs. Multi-criteria decision analysis (MCDA) has been successfully used to increase effectiveness in various fields by supporting decision-making. However, no review of its application to all-hazard health emergencies has been done to date. A review of peer-reviewed English-language articles published since 2004 was conducted in May 2024 using Scopus, PubMed and Web of Science databases. The review focused on the empirical application of MCDA to support decision-making during health emergencies. The review was guided by the Joanna Briggs Institute methodology for scoping reviews and adhered to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews. Quantitative data were analyzed using summary statistics and qualitative data were analyzed using content analysis. Seventy-one articles were included after screening. The articles described the MCDA application to support a variety of decision problems related to health emergency management. However, the technique was mostly applied to infectious hazards management and only seldom to other hazards. The review also found a lack of standardized methodology for identifying alternatives and criteria, weighting, computation of model output, methods of dealing with uncertainty, and stakeholder engagement. The review provides an overview of the current use of the MCDA approach to support decision-making in health emergency management and informs areas of future development. The review emphasizes that while MCDA is already used for infectious hazards, it is underutilized for other types of health emergencies. Developing tailored MCDA approaches for health emergencies, including defining evaluation criteria and stakeholder engagement, may improve uptake of the technique and benefit the efforts to meet the growing health needs of the population affected by emergencies, https://osf.io/6kd5s/.

Use of multi-criteria decision analysis to identify potentially dangerous glacial lakes

Reviewing the use of Multi-Criteria Decision Analysis for the evaluation of transport projects: Time for a multi-actor approach