Preference Elicitation Process Research Articles

5613464 INTEGRATION OF PHARMACOKINETIC AND PHARMACODYNAMIC MODELS TO MULTI-CRITERIA DECISION ANALYSIS (MCDA) FOR PROSPECTIVE ASSESSMENT OF THE RISK-BENEFIT RATIO OF ORAL IRON CHELATORS IN PAEDIATRIC PATIENTS WITH TRANSFUSION-DEPENDENT HAEMOGLOBINOPATHIES

Background: Current therapies for the treatment of thalassaemia and other hereditary rare haemoglobinopathies are symptomatic and aim at controlling or reducing iron overload by chelation of iron from a very early age. Iron chelation ultimately prevents complications or comorbidities that can significantly reduce patient survival. Given the processes underlying iron homeostasis and current understanding of the chelation mechanism of available chelating agents, it may be possible to implement alternative, more efficient approaches for evidence generation in support of the dose rationale and their use across different populations affected by transfusion-dependent haemoglobinopathies. Aim: This study aimed to characterise the benefit-risk balance (BRB) of deferiprone in different populations, including children < 2 years old and paediatric sickle cell patients, using a nonlinear mixed effects modelling approach in conjunction extrapolation concepts and multicriteria decision analysis (MCDA). Methods: Based on the underlying pharmacokinetic-pharmacodynamic relationships as well as on the incidence of adverse events, extrapolation concepts were applied to predict the BRB of treatment in young adults, paediatric patients younger than 2 years, and paediatric sickle cell disease patients receiving deferiprone. Using individual patient-level data from a randomised controlled clinical trial (DEEP-2, EudraCT 2012-000353-31), preference elicitation methods were applied to select and prioritise endpoints or outcomes to be used in the evaluation of the BRB across the populations of interest. Subsequently, model parameter estimates obtained from each model/endpoint were used in a set of clinical simulation scenarios in which the effect of chelation therapy was evaluated during 12 months in a virtual cohort of patients. Results: Five endpoints were selected and ranked by the experts involved in the preference elicitation process (Fig.1). ‘Absence or prevention of complications (diabetes/hypothiroidism)’ and ‘occurrence of agranulocytosis’ were the most important criteria, while ‘incidence of arthralgia’ and ‘occurrence of neutropenia’ were considered less important. Clinical trial simulations revealed that the overall benefit score of deferiprone was higher for the treatment of paediatric SCD patient compared with the reference population (i.e. DEEP-2 study). Small but statistically significant differences were observed for the use of DFP in children younger than 2 years old and adolescents or young adults. Table 1 summarises the clinical and demographic baseline characteristics of the DEEP-2 study and virtual cohort of patients included in the different simulation scenarios. Conclusion: This study has shown that MCDA can be utilised in combination with modelling and simulation principles to prospectively assess the effect of interventions in subgroups of patients for which clinical data generation is challenging or not feasible. It also offers an opportunity to evaluate the implications of dose adjustment as well as changes in the clinical management of these patients, providing a less empirical basis for dose selection and treatment choices in rare paediatric conditions such as transfusion-dependent haemoglobinopathies. Table 1 - Demographics of the populations in the MCDA analysis. Reference population reflects the actual patients in the DEEP-2 study, whilst scenarios reflect the baseline characteristics from the virtual patient population. Population Age (years) Weight (kg) Blood consumption (mL/kg/year) Baseline ferritin (ng/mL) Reference population (DEEP-2 study) 8.9 (2.7-17.2) 26.5 (12.1-53.5) 155.4 (17.6-404.3) 1991 (586.2-6488.7) Scenario 1: Adolescents and young adults 17.6 (11.3-33.8) 58 (29-83.6) 139.7 (28.5-347.6) 1933 (432-10106) Scenario 2: Children < 2 years old 1.3 (0.6-1.9) 6.3 (1.1-9.3) 162.2 (21-382.9) 726.3 (253.3-1757) Scenario 3: Paediatric sickle cell patients 14.5 (12.4-17.8) 39 (30-62.4) 139.5 (23.8-338.2) 1598.4 (476.3-8993.9)

Patient Decision Aids for Values Clarification and Preference Elicitation - Challenges and Developments

Patient Decision Aids for Values Clarification and Preference Elicitation - Challenges and Developments Abstract. Shared decision-making is especially appropriate when the available evidence does not indicate which medical intervention is the better option, so that the final decision depends on the patient's personal values and preferences. The process of value clarification and preference elicitation can be time-consuming and cognitively and emotionally demanding for patients. Increasingly, decision aids provide tasks (e.g., on benefit-harm trade-offs) to help patients work through this process, better prepare for medical consultations, and make values-congruent medical decisions with their physicians. Most clinically validated decision aids are paper-based flyers and educational brochures. There are also computer-, audio-, video-, or web-based decision aids. The web-based aids make little use of the potential of interactive technologies, despite the known benefits of these technologies. The aims of this paper are to provide an overview of decision aids for and challenges of values clarification and preference elicitation and to highlight some developments in interactive web-based technologies that might facilitate values clarification and preference elicitation.

Analysis of the cognitive aspects of the preference elicitation process in the compensatory context: a neuroscience experiment with FITradeoff

AbstractMulti‐criteria methods and systems have been developed over many years to support decision‐making with a process of eliciting preferences. The emphasis has been on those that incorporate compensatory rationality, the characteristics of which make the process more complex. Neuroscience has offered the opportunity to capture aspects of cognition, generating insights. This paper investigates the preference elicitation process with Flexible and Interactive Tradeoff (FITradeoff), using electroencephalography and eye‐tracking applied to a sample of undergraduate and graduate students. While FITradeoff offers greater flexibility when evaluating recent advances in the area, the experiment carried out incorporates paradigms present in other methods developed for compensatory rationality. The results indicate the effect the type of predominant criterion has on cognitive effort. Moreover, a non‐linear relationship between the number of criteria and questions was found, which suggests that decision‐makers use a strategy of minimizing cognitive effort. Such findings help to direct efforts to enhance systems and methods with compensatory rationality.

Interactive decision support and trade-off analysis for sustainable forest landscape planning under deep uncertainty

Sustainable environmental management often involves long-term time horizons and multiple conflicting objectives and, by nature, is affected by different sources of uncertainty. Many sources of uncertainty, such as climate change or government policies, cannot be addressed using probabilistic models, and, therefore, they can be seen to contain deep uncertainty. In this setting, the variety of possible future states is represented as a set of scenarios lacking any information about the likelihood of occurring. Integrating deep uncertainty into multiobjective decision support increases complexity, calling for the elaboration of appropriate methods and tools. This paper proposes a novel interactive multi-scenario multiobjective approach to support decision-making and trade-off analysis in sustainable forest landscape planning under multiple sources of uncertainty. It includes new preference simulation models aimed at reducing the decision-maker’s cognitive load and supporting the preference elicitation process. The proposed approach is applied in a case study of long-term forest landscape planning with four sustainability objectives in 12 scenarios and a forestry expert as the decision-maker. The approach is demonstrated to be efficient in exploring trade-offs in different scenarios, helping the expert gain deep insights into the problem, understand the consequences of alternative strategies, and find the most preferred robust strategy.

Iterative Multi-Attribute Procurement Auction with Decision Support for Bid Formulation

Iterative multi-attribute reverse auctions in practice create certain difficulties for both the buyer and participating bidders. While the buyer faces the problem of creating the right attribute weights, the bidders have difficulty in adjusting the attribute values in each round. In this paper, we present an iterative multi-attribute reverse auction mechanism based on integrated data envelopment analysis (DEA) and best–worst method (BWM) with an objective of reducing the intervention of the buyer in the determination of the winner and also easing up the preference elicitation process. Unlike the typical scoring auctions, the proposed mechanism does not require the buyer to estimate the characteristics of the participating sellers in order to determine the optimal scoring function. As there will be no other intervention from the buyer during the winner determination process, the proposed method makes the procurement process impartial and corruption-free. Besides solving the buyer’s problem, the proposed mechanism is also associated with an optimal bid determination method (OBDM) to assist the sellers in formulating improvised bids in iterative rounds of the auction. Simulation experiments show that the proposed OBDM benefits both the buyer and sellers. For the buyer, it provides higher expected utility and attribute values as per his preferences; for the seller, it gives a better expected profit and a higher probability of winning.

A Survey on Conversational Recommender Systems

Recommender systems are software applications that help users to find items of interest in situations of information overload. Current research often assumes a one-shot interaction paradigm, where the users’ preferences are estimated based on past observed behavior and where the presentation of a ranked list of suggestions is the main, one-directional form of user interaction. Conversational recommender systems (CRS) take a different approach and support a richer set of interactions. These interactions can, for example, help to improve the preference elicitation process or allow the user to ask questions about the recommendations and to give feedback. The interest in CRS has significantly increased in the past few years. This development is mainly due to the significant progress in the area of natural language processing, the emergence of new voice-controlled home assistants, and the increased use of chatbot technology. With this article, we provide a detailed survey of existing approaches to conversational recommendation. We categorize these approaches in various dimensions, e.g., in terms of the supported user intents or the knowledge they use in the background. Moreover, we discuss technological approaches, review how CRS are evaluated, and finally identify a number of gaps that deserve more research in the future.

A comparative analysis of probabilistic linguistic preference relations and distributed preference relations for decision making

When a decision-maker prefers to compare different alternatives in pairs to handle real situations, there are many different expression styles that can be used. Two representative expression styles are the probabilistic linguistic preference relation (PLPR), which originates from the fuzzy linguistic approach and the distributed preference relation (DPR), which originates from the evidential reasoning approach. Although these two expression styles look quite similar, their meanings, operations, and relevant decision making processes are significantly different. This presents the decision-maker with the challenge of selecting either PLPRs or DPRs in different real cases. To address this issue, this paper provides a detailed analysis of the similarities and differences between PLPRs and DPRs. The analysis is conducted from five perspectives, including modeling of decision making problems, handling of uncertainty, consistency between preference relations, information aggregation, and elicitation process. An engineer selection problem for an automobile manufacturing enterprise is investigated to demonstrate how to appropriately select PLPRs or DPRs to model and analyze decision making problems in real situations with consideration for the preferences of decision-makers.

Multi-dimensional stability analysis for Analytic Network Process models

The decision-making process involves, most of the time, working with limited and/or incomplete information to obtain a one-time synthesis of one’s preferences. However, additional information might be acquired by the decision-maker at a later time. To identify the impact that the additional information could have on the initial result, we developed a new multi-dimensional stability analysis that provides a comprehensive image of how preferences change or evolve, as perturbations are applied to the criteria weights, without reiterating the preference elicitation process. The insights provided by the new method helped define a set of stability measures useful in a practical setting. To show the direct implementation of the findings of the multi-dimensional stability analysis, we applied the method developed to a randomly generated ANP model.

Lie on the Fly: Strategic Voting in an Iterative Preference Elicitation Process

A voting center is in charge of collecting and aggregating voter preferences. In an iterative process, the center sends comparison queries to voters, requesting them to submit their preference between two items. Voters might discuss the candidates among themselves, figuring out during the elicitation process which candidates stand a chance of winning and which do not. Consequently, strategic voters might attempt to manipulate by deviating from their true preferences and instead submit a different response in order to attempt to maximize their profit. We provide a practical algorithm for strategic voters which computes the best manipulative vote and maximizes the voter's selfish outcome when such a vote exists. We also provide a careful voting center which is aware of the possible manipulations and avoids manipulative queries when possible. In an empirical study on four real-world domains, we show that in practice manipulation occurs in a low percentage of settings and has a low impact on the final outcome. The careful voting center reduces manipulation even further, thus allowing for a non-distorted group decision process to take place. We thus provide a core technology study of a voting process that can be adopted in opinion or information aggregation systems and in crowdsourcing applications, e.g., peer grading in Massive Open Online Courses (MOOCs).

On preference elicitation processes which mitigate the accumulation of biases in multi-criteria decision analysis

In the practice of multi-criteria decision analysis, biased responses to the preference elicitation questions may impact the outcome of the process. In particular, there is a risk that the effects of biases accumulate in favor of a single alternative or a subset of alternatives. In this paper, we develop new bias mitigation techniques for multi-criteria decision analysis, which are based on the idea that the effects of biases can cancel out each other in the preference elicitation process. The benefits of the techniques include that the decision maker does not need to try to change her behavior to avoid biases, and there are no numerical adjustments of her judgements. The new techniques that we propose are: (1). Introducing a virtual reference alternative in the decision problem. (2). Introducing an auxiliary measuring stick attribute. (3). Rotating the reference point. (4). Restarting the decision process at an intermediate step with a reduced set of alternatives. We simulate computationally how these techniques help mitigate biases in the Even Swaps process when the decision maker exhibits the loss aversion bias, the measuring stick bias, and makes random response errors. The techniques can also be applied in weight elicitation using the SWING and trade-off methods to reduce the aforementioned biases.

A Human-Computer Interface Design for Quantitative Measure of Regret Theory

Regret theory is a theory that describes human decision-making under risk. The key of obtaining a quantitative model of regret theory is to measure the preference in humans' mind when they choose among a set of options. Unlike physical quantities, measuring psychological preference is not procedure invariant, i.e. the readings alter when the methods change. In this work, we alleviate this influence by choosing the procedure compatible with the way that an individual makes a choice. We believe the resulting model is closer to the nature of human decision-making. The preference elicitation process is decomposed into a series of short surveys to reduce cognitive workload and increase response accuracy. To make the questions natural and familiar to the subjects, we follow the insight that humans generate, quantify and communicate preference in natural language. The fuzzy-set theory is hence utilized to model responses from subjects. Based on these ideas, a graphical human-computer interface (HCI) is designed to articulate the information as well as to efficiently collect human responses. The design also accounts for human heuristics and biases, e.g. range effect and anchoring effect, to enhance its reliability. The overall performance of the survey is satisfactory because the measured model shows prediction accuracy equivalent to the revisit-performance of the subjects.

Multi Criteria Decision Making in Selecting Stormwater Management Green Infrastructure for Industrial Areas Part 1: Stakeholder Preference Elicitation

Green Infrastructure (GI) practices are well known strategies which are used for the source control of stormwater generated from urban land uses. However, unlike other urban land uses, industrial areas are land use types which need special attention in selecting the most suitable GI among several practices available for the stormwater management due to the certain environmental considerations. The decision making in selecting the stormwater management GI practices for an industrial area involves environmental, economic and social objectives which are represented by various performance measures. When identifying these performance measures, the opinions of stakeholders play a major role for the successful implementation of GI. This paper (part 1 of 2 part series of papers) presents the results of a stakeholder preference elicitation process carried out through a four rounded Delphi survey to identify the performance measures and obtain their weights for the decision making of GI for industrial areas. The SWING weighting method was embedded within the Delphi survey process to elicit the weights for various performance measures. The experts have identified the total runoff volume reduction as the most important performance measure in selecting a suitable GI for industrial areas. The results obtained from the Delphi survey series showed a good degree of consensus and has proven to be a valuable tool for the decision making in GI selection, for industrial areas. The stakeholder preference elicitation presented in this paper is a part of a Multi Criteria Decision Analysis (MCDA) conducted to identify the most suitable stormwater management GI practices for an industrial area.

PP43 MACBETH In Brazilian Hospital-Based HTA: Thrombosis Prophylaxis

Introduction:Instituto Nacional de Traumato-Ortopedia (INTO) administrates the Enoxaparin drug to prevent deep vein thrombosis (DVT) after extensive orthopedic surgeries. Nevertheless, new oral anticoagulants that offer more comfort and efficacy, but present higher risk of bleeding, have been putting in question the use of Enoxaparin. Making use of the MACBETH method, this study develops a Multicriteria Value Measurement model to evaluate such drugs.Methods:MACBETH was applied in helping INTO to evaluate two drugs (Rivaroxaban and Enoxaparin), taking into account drug benefits and risks, through a series of interviews and decision conferences attended by INTO stakeholders that acted as evaluators in the model-building process, supported by M-MACBETH DSS (www.m-macbeth.com). Following MACBETH preference elicitation process, the evaluators were asked to make qualitative pairwise comparison judgements of difference in value between stimuli for constructing quantitative value and weighting scales. These scales allow measuring the relative value of the drugs on each evaluation criterion, separately and globally. The value measurement process was informed by a literature review and meta-analysis of randomized clinical trials with a critical appraisal of the evidence.Results:We report a model-structure with eight criteria, hereafter presented by decreasing order of their weighting: Death from any cause, Clinically significant bleeding, Proximal DVT, Distal DVT, Existence of antidote, Thrombocytopenia, Costs, and Comfort. From the value model developed and after performing sensitivity and robustness analyses, Rivaroxaban was considered a robust option for thrombosis prophylaxis, under the MACBETH value framework and at the light of a simple additive aggregation of those eight criteria.Conclusions:This study shows how a value measurement socio-technical framework, combining MACBETH with scientific evidence within a participatory group evaluation process, can support health technology assessment in a user-friendly and effective way. MACBETH facilitates transparent and robust decision-making in the face of complex evaluation problems that the hospital often faces.

Selection of a faculty member in academia: a case for verbal decision analysis

Personnel selection problem is often viewed as a multi criteria problem and there are many examples when different multi criteria techniques support the process. The multi criteria approach was used in several studies of faculty selection in academia but not many recognised that there were peculiarities in the selection process. Most studies concentrated on rank ordering of a small number of finalists while one of the main tasks of the search committee in academia is to select a smaller number of candidates from a possibly large pool of applicants for a more thorough review. In our paper we propose a three-step approach to the problem and recommend verbal decision analysis (VDA) as a tool. Using a VDA method provides a reliable preference elicitation process and the decision rules are easy to understand and explain. Free tool ORCON_Z helps with the practical application. The proposed procedure was well received by the participants and provided valuable information during the selection process.

O estymacji preferencji w sztucznych sieciach społecznych

The paper considers a scenario in which public administration (PA) uses an online social platform to collect information on citizens' preferences. However, the opinions of the sub-population that uses the online platform might be not representative. The author develops a method for generalization of the dynamics of the preferences observed on the social platform onto the entire population. The available data include information collected by the PA from the online platform (assuming that it is run and administered by the PA) and census data regarding the population. Hence, the PA has access to basic personal data of platform users (e.g. gender and age), position in the online social network, and opinions revealed on the platform. The online users' data can be analyzed along with the aggregated census data on the entire population. The author has implemented a multi-agent simulation model that takes into account the distribution of personal attributes, social network data, and opinion diffusion dynamics. The analysis involves showing how different algorithms enable generalization of preferences collected by the online platform to the entire population. The results of the analysis prove that the proposed method is efficient in the preference elicitation process – with each simulation step, the preference congruence level between real and synthetic populations increases. The main determinants of preference elicitation errors include the preference diffusion model and the weight of the agents’ own opinions.

The MARS Approach in the Verbal and Holistic Evaluation of the Negotiation Template

The negotiation template, which defines a set of potential negotiation offers, is traditionally evaluated by means of the simple additive weighting method (SAW). However, some recent research reports on the potential problems and inconsistencies in using and interpreting SAW-based scores. Thus, in this paper we consider the issue of evaluating negotiation offers when the negotiator’s preferences are expressed verbally. We present a new approach called Measuring Attractiveness near Reference Situations (MARS), which combines the algorithms of two multiple criteria decision making methods: ZAPROS and MACBETH. Applying the elements of ZAPROS allows identifying a small set of reference alternatives that consists of the best resolution levels for all the negotiation issues but one. In pair-wise comparisons of these alternatives negotiators need to evaluate trade-offs only, which means deciding which concessions are better to be made. Using the elements of MACBETH allows determining the strong interval scale based on verbal judgments defined by negotiators at the beginning of the preference elicitation process. We study in detail the legitimacy of hybridizing ZAPROS and MACBETH that differ in their philosophies of decision support as well as discuss the drawbacks of these two MCDM methods and propose some alternative solutions that make this approach applicable to supporting negotiators in the evaluation of negotiation offers. Finally, we present an example in which we indicate the differences in the negotiation offers’ scoring process conducted by means of MARS and the traditional ZAPROS and MACBETH procedures.

Value-Based Assessment of New Medical Technologies: Towards a Robust Methodological Framework for the Application of Multiple Criteria Decision Analysis in the Context of Health Technology Assessment.

In recent years, multiple criteria decision analysis (MCDA) has emerged as a likely alternative to address shortcomings in health technology assessment (HTA) by offering a more holistic perspective to value assessment and acting as an alternative priority setting tool. In this paper, we argue that MCDA needs to subscribe to robust methodological processes related to the selection of objectives, criteria and attributes in order to be meaningful in the context of healthcare decision making and fulfil its role in value-based assessment (VBA). We propose a methodological process, based on multi-attribute value theory (MAVT) methods comprising five distinct phases, outline the stages involved in each phase and discuss their relevance in the HTA process. Importantly, criteria and attributes need to satisfy a set of desired properties, otherwise the outcome of the analysis can produce spurious results and misleading recommendations. Assuming the methodological process we propose is adhered to, the application of MCDA presents three very distinct advantages to decision makers in the context of HTA and VBA: first, it acts as an instrument for eliciting preferences on the performance of alternative options across a wider set of explicit criteria, leading to a more complete assessment of value; second, it allows the elicitation of preferences across the criteria themselves to reflect differences in their relative importance; and, third, the entire process of preference elicitation can be informed by direct stakeholder engagement, and can therefore reflect their own preferences. All features are fully transparent and facilitate decision making.

Using Multiple Criteria Decision Analysis (MCDA) to make coverage decisions: a methodological framework

Results We propose a methodological process comprising five distinct phases (problem structuring, model building, model assessment, model appraisal and development of action plans), outline the stages involved in each phase and discuss their relevance in the HTA process. Additionally, criteria and attributes need to satisfy a set of desired properties, otherwise the process of preference elicitation can produce spurious results. The resulting MCDA output, which can take the form of a single universal value index, can be informed by stakeholder participation and therefore can be robust and reflective of stakeholder preferences.

Reaching a joint decision with minimal elicitation of voter preferences

Sometimes voters are required to reach a joint decision and find an item that best suits the group’s preferences. Voters may wish to state preferences only when necessary, particularly in cases where there are many available options, therefore it is unpractical to assume that all voter preferences are known at all times. In order to elicit voter preferences at a minimal cost, a preference elicitation process is required. We introduce a general approach for reaching a joint decision with minimal elicitation of voter preferences. The approach is probabilistic and uses voting rules to find a necessary winning item which is presented to the group as their best option. We propose computing a voter-item probability distribution and developing methods based on this distribution that can then determine which voter-item pair to query. Computing the optimal minimal set of voter-item queries is computationally intractable; therefore we propose novel heuristic algorithms, named DIG and ES, which proceed iteratively until the identification of a winning item. The probabilistic voting distribution is updated as more information is revealed. Experiments on simulated data examine the benefits of each of the algorithms under different settings. Experiments with the real-world Netflix data show that the proposed algorithms reduce the required number of ratings for identifying the winning item by more than 50%.

An Approach to Multi‐Criteria Decision Problems Under Severe Uncertainty

ABSTRACTSolving complex decision problems is a demanding task; it requires determining and evaluating the consequences of decision alternatives. To this end, uncertain factors that can only partly be influenced by the decision makers, and their interdependencies need to be considered. Scenarios focus on this part of the decision problem; they enable a systematic exploration of a multitude of possible future developments that are relevant for the decision including external events and decisions made. Scenarios are particularly useful when the problem is pervaded by severe uncertainties that cannot be quantified. For the evaluation of alternatives, multiple objectives and the potentially diverging preferences of the involved actors need to be respected. Multi‐criteria decision analysis aims at structuring the problem, evaluating the alternatives and supporting decision makers pursuing multiple goals.We propose an approach integrating scenarios and multi‐criteria decision analysis that focuses on the robustness of alternatives in complex, dynamic, uncertain and time‐bound situations. In this integrated framework, the scenarios provide the basis for evaluating a set of alternatives. Ideally, the set of scenarios considered captures all possible future developments. To appropriately explore this set, formal or analytical approaches to scenario construction generate a large number of scenarios. This challenges the decision makers' information‐processing capacity. To support them in managing the richness of information, a two‐fold approach that uses selection and aggregation is presented. By using a selection method, the scenarios that are deemed most relevant are identified, and their evaluations are presented in detail to decision makers. This approach is complemented by an aggregation of scenario evaluations on the basis of the decision makers' preferences. We present two approaches to facilitate the preference elicitation process. Copyright © 2013 John Wiley &amp; Sons, Ltd.