Multi-criteria Decision Analysis Process Research Articles

Using a Patient-Centered Multicriteria Decision Analysis to Assess the Value of Multiple Sclerosis Treatments in the US: A Study Protocol.

The engagement of patients and family caregivers in value assessment is pivotal since they provide valuable contributions to assessment acceptability and relevance. The proposed study aims to use patient-centered techniques and multicriteria decision analysis (MCDA) to evaluate the values of disease-modifying therapies (DMTs) for multiple sclerosis (MS) from the perspectives of patients and family caregivers living in three 'Deep South' States of the US-Alabama, Louisiana, and Mississippi. This study will follow guidance from the Patient-Centered Outcomes Research Institute (PCORI) for patient engagement and two best practice reports for MCDA from the Professional Society for Health Economics and Outcomes Research (ISPOR) to complete value assessment. Throughout the study, we will engage multiple stakeholders, including patients, family caregivers,healthcare providers, and payers. Forty patients with MS and their family caregivers from Alabama, Louisiana, and Mississippi will be invited to participate in this study. We will intensively train them for value assessment knowledge and MCDA before we engage them in MCDA to determine the value of DMTs for MS. Our approach differs from common MCDA since we incorporated a patient-centered framework in this study. Unlikeprevious studies only briefly inform or prepare participants before the MCDA process, in this study, we will provide basicvalue assessment trainings forpatients and family caregivers to ensure they can effectively engage throughout the patient-centeredMCDAprocess. We expect this study will demonstrate that the patient-centered MCDA approach is feasible and likely leads to improved patients' and family caregivers' engagementin value assessment.

Development of a Choice-framework for Covid vaccines in India using a multi-criteria decision analysis approach

BackgroundVaccines were crucial in controlling the Covid-19 pandemic. As more vaccines receive regulatory approval, stakeholders will be faced with several options and must make an appropriate choice for themselves. We proposed a multi-criteria decision analysis (MCDA) framework to guide decision-makers in comparing vaccines for the Indian context. MethodsWe adhered to the ISPOR guidance for the MCDA process. Seven vaccine options were compared under ten criteria. Through three virtual workshops, we obtained opinions and weights from citizens, private-sector hospitals, and public health organisations. Available evidence was rescaled and incorporated into the performance matrix. The final score for each vaccine was calculated for the different groups. We performed different sensitivity analyses to assess the consistency of the rank list. ResultsThe cost, efficacy and operational score of the vaccines had the highest weights among the stakeholders. From the six scenario groups, Janssen had the highest score in four. This was driven by the advantage of having a single dose of vaccination. In the probabilistic sensitivity analysis for the overall group, Covaxin, Janssen, and Sputnik were the first three options. The participants expressed that availability, WHO approvals and safety, among others, would be crucial when considering vaccines. ConclusionsThe MCDA process has not been capitalised on in healthcare decision-making in India and LMICs. Considering the available data and stakeholder preference at the time of the study, Covaxin, Janssen, and Sputnik were preferred options. The choice framework with the dynamic performance matrix is a valuable tool that could be adapted to different population groups and extended based on increasing vaccine options and emerging evidence.*ISPOR – The Professional Society for Health Economics and Outcomes Research.

Quantitative Benefit-Risk Assessment in Medical Product Decision Making: A Good Practices Report of an ISPOR Task Force

Benefit-risk assessment is commonly conducted by drug and medical device developers and regulators, to evaluate and communicate issues around benefit-risk balance of medical products. Quantitative benefit-risk assessment (qBRA) is a set of techniques that incorporate explicit outcome weighting within a formal analysis to evaluate the benefit-risk balance. This report describes emerging good practices for the 5 main steps of developing qBRAs based on the multicriteria decision analysis process. First, research question formulation needs to identify the needs of decision makers and requirements for preference data and specify the role of external experts. Second, the formal analysis model should be developed by selecting benefit and safety endpoints while eliminating double counting and considering attribute value dependence. Third, preference elicitation method needs to be chosen, attributes framed appropriately within the elicitation instrument, and quality of the data should be evaluated. Fourth, analysis may need to normalize the preference weights, base-case and sensitivity analyses should be conducted, and the effect of preference heterogeneity analyzed. Finally, results should be communicated efficiently to decision makers and other stakeholders. In addition to detailed recommendations, we provide a checklist for reporting qBRAs developed through a Delphi process conducted with 34 experts.

Astronomical observatory site selection using fuzzy AHP and BWM methods

Establishing an observatory often involves complex decisions, such as choosing a site based on multiple conflicting criteria. In this study, we develop a multi-criteria decision analysis process by combining Geographic Information System (GIS) analysis with the (MCDA) Multi-Criteria Decision Analysis and use this process to determine the most suitable sites for the construction of an observatory in the Malatya urban area. GIS was used to calculate, classify, and analyze criteria, while FAHP (Fuzzy Analytic Hierarchy Processes, Buckley's method) and BWM(MCDA type Best-Worst Method) methods were used to weight the decision criteria and determine their effects on alternative sites. While the Cloud Cover criterion in the BWM method was the most important criterion with 28%, the most important criterion for the FAHP method had a comparable value of 27.8%. Meteorological criteria were the most important criteria group with values of 50.4% according to the FAHP method and 44.6% according to the BWM method. The study is based on meteorological, geographic, and anthropogenic datasets, suggesting the most appropriate sites for the astronomical observatory within the boundaries of the study area. The proposed sites are the result of site selection, which is the first phase of site selection for astronomical observatories. This site selection is important to limit the number of field alternatives. It is necessary to conduct field tests among the proposed areas and select the final site according to the results. The successful use of GIS and more than one MCDA method will pave the way for the development of various methods for astronomical observatory site determination.

Alternative Marine Fuel Research Advances and Future Trends: A Bibliometric Knowledge Mapping Approach

While the marine sector contributes significantly to the global economy, its environmental impact is a cause for apprehension due to growing concerns about ship emissions. The International Maritime Organization (IMO) has set decarbonization strategies consistent with sustainable development goals. The impending legislation aimed at reducing greenhouse gas (GHG) emissions from maritime shipping by at least half by 2050 and to zero by the end of the century. A growing body of research has focused on alternative marine fuel selection; hence, this bibliometric review is timely. We assess the global scientific research on alternative marine fuel for knowledge mapping based on the articles available on the Scopus database since 1973. A total of 749 publications associated with alternative marine fuel has been subjected to a range of bibliometric analyses to explore this research field quantitatively and qualitatively. The study utilized the R-studio bibliometrics package and VOSviewer bibliometric tools to generate the results. The field of study has a growth rate of 7.05%, having a significant contribution to knowledge from the USA, the United Kingdom, India, and China. Recent trends indicate that researchers are increasingly focused on alternative fuel oil choices. By analyzing commonly used keywords, it was possible to trace a multi-criteria decision analysis process that might be utilized to construct decision support systems for alternative maritime fuel selection. According to relevant articles, the research community has concentrated on the possibility of alternative fuels being utilized in place of traditional marine fuels to reduce emissions from the shipping sector from an environmental, technological, and economic standpoint. The current study offers an updated and comprehensive overview of research trends on alternative marine fuels. Researchers interested in data mapping work in this area can begin with this endeavor.

Decision analysis support for evaluating transmission risk of COVID-19 in places where people gather.

The coronavirus diseases 2019 (COVID-19) pandemic has presented an unprecedented public health challenge. Prior to vaccination, non-pharmaceutical interventions, including closures, were necessary to help control the epidemic. With the arrival of variants of concern and insufficient population vaccination coverage, ongoing evaluation of transmission risk in settings and the use of non-pharmaceutical interventions are necessary to help control the epidemic. This study aimed to produce a framework for evaluating transmission risk in settings where individuals gather and inform decision-making. A multi-criteria decision analysis process was used to structure the framework. Fifteen criteria were identified as important to consider for COVID-19 transmission risk based on the literature. This list was ranked by experts and then categorized. The analysis was structured by the consensus list of criteria and relative positioning of each criteria within the list to produce sets of factors to consider when assessing transmission risk at gatherings. Fifteen experts from across Canada participated in ranking the criteria. Strong consensus was found on the relative importance of criteria and this relative consensus was used to create four categories: critical (3 criteria); important (6 criteria); good to consider (5 criteria); and if time permits (1 criterion). The resulting consensus list and categories constitutes a set of important elements that can be applied to any setting as an objective and transparent framework to assess transmission risk in the venue. In conjunction with further consideration of the local epidemiology of COVID-19, an overall risk of transmission assessment can be established and uniformly implemented.

A coupled optimization of groundwater remediation alternatives screening under health risk assessment: An application to a petroleum-contaminated site in a typical cold industrial region in Northeastern China

Contaminated sites have been recognized as posing serious comprehensive social and environmental issues and have earned worldwide attention. China is becoming one of the largest contaminated sites remediation markets in the world and the contaminated sites in northeastern China need to rehabilitate urgently. However, remediation planning is often hindered by high financial costs resulting from incomplete assessments of pollution and inappropriate remediation plans. In-depth contaminated site assessments can provide the necessary baseline data for remediation alternatives screening. Therefore, risk assessments and remediation decisions will play crucial roles in the rehabilitation and reconstruction of contaminated sites in China. The main objectives of this study were to present a novel method for health risk assessment (HRA) and to demonstrate a multicriteria decision analysis (MCDA) based on this method to select the most suitable remediation alternatives of groundwater and to prioritize management of contaminated site. To demonstrate the HRA and MCDA processes, a typical contaminated site in Longtan, Jilin province, China, was used. The results of this research indicated that Benzene (PhH) and 1,2-Dichloroethylene (1,2-DCE) were the main organic pollutants and the vanillin plant in the north of the site was main pollution source. Pollution migrated from the north to the south and the health risk range in winter was significantly greater than in summer. Four remediation alternatives were proposed on the basis of the HRA results. The MCDA results showed that PRB was the most suitable technology for integrating the relevant environmental, social, economic, and technical aspects required for remediation. This study may help responsible agencies to strengthen local risk-based program screening frameworks for contaminated sites, to promote reconstruction projects, and to increase local public confidence of contaminated sites remediation.

Identification of Relevant Criteria Set in the MCDA Process—Wind Farm Location Case Study

The paper undertakes the problem of proper structuring of multi-criteria decision support models. To achieve that, a methodological framework is proposed. The authors’ framework is the basis for the relevance analysis of individual criteria in any considered decision model. The formal foundations of the authors’ approach provide a reference set of Multi-Criteria Decision Analysis (MCDA) methods (TOPSIS, VIKOR, COMET) along with their similarity coefficients (Spearman correlation coefficients and WS coefficient). In the empirical research, a practical MCDA-based wind farm location problem was studied. Reference rankings of the decision variants were obtained, followed by a set of rankings in which particular criteria were excluded. This was the basis for testing the similarity of the obtained solutions sets, as well as for recommendations in terms of both indicating the high significance and the possible elimination of individual criteria in the original model. When carrying out the analyzes, both the positions in the final rankings, as well as the corresponding values of utility functions of the decision variants were studied. As a result of the detailed analysis of the obtained results, recommendations were presented in the field of reference criteria set for the considered decision problem, thus demonstrating the practical usefulness of the authors’ proposed approach. It should be pointed out that the presented study of criteria relevance is an important factor for objectification of the multi-criteria decision support processes.

Does the spatial representation affect criteria weights in environmental decision-making? Insights from a behavioral experiment

The need and interest to consider cognitive and motivational biases has been recognized in different disciplines (e.g. economics, decision theory, risk analysis) and has recently reached environmental decision-making. Within this domain, the intrinsic presence of a spatial dimension of both alternatives and criteria calls for the use of maps throughout the decision-making process to properly represent the spatial distribution of the features under analysis. This makes spatial Multi Criteria Decision Analysis (MCDA) a particularly interesting domain to explore new dimensions of cognitive biases. This study proposes a behavioral experiment aimed at discovering to what extent the spatial visualization (i.e. maps) of criteria versus the non-spatial one (i.e. tables) can bias the weight elicitation phase of a spatial MCDA process. The experiment simulates a very common analysis in environmental and land use planning: land suitability analysis. Our findings show that there are significant consequences on how important we perceive a certain criterion to be, depending on whether it is represented as a map or as a table among a mix of maps and tables. Indeed, the map representation of the same criterion leads to higher weights attributed to that criterion compared to the table representation. Visualizing the same information as a map or as a table, although technically equivalent, is thus not psychologically equivalent for Decision Makers. The results of this experiment are expected to have implications for spatial decision-making processes, by generating better awareness on the impacts of map-mediated land suitability analysis.

Decision analysis for sustainable development: The case of renewable energy planning under uncertainty

Multi-Criteria Decision Analysis (MCDA) methods are increasingly used to aid decision-making for sustainable development. However, although uncertainty is present in all decision environments, dealing with incomplete and vague information in decision analysis remains a challenge. Our objective is to simplify the incorporation of uncertainty in the scoring of alternatives in MCDA processes. We present a modified ELimination and Choice Translating REality (ELECTRE) III model, in which the uncertainty in the performance scores is expressed as lower/upper bounds and then added to the model’s discrimination thresholds. Unlike other uncertainty approaches developed in the literature (such as those based on fuzzy set theory), our approach does not require additional knowledge apart from understanding the ELECTRE III model. To test its validity and suitability, we apply it for the evaluation of renewable energy resources for Turkey – hydro, wind, geothermal, solar, and biomass – under five main criteria: technological, technical, economic, environmental, and socio-politic. Our results indicate that wind energy is the best alternative for Turkey, followed by solar energy, which is in line with country’s Vision 2023 energy targets.

A blueprint for a methodology to balance land-use in semi-natural areas

I investigated the effectiveness of a multi-criteria decision analysis process in integrating the study of community composition, aimed to allow sustainable use of semi-natural habitats. Weighted linear combination identified the suitability for recreational use and conservation potential of different sectors of the study area, obtained respectively from questionnaires and the examination of plant biodiversity. After assessing human impacts, I created a compromise scenario using a multi-objective land allocation method. I found a positive relationship between the most frequented areas and the presence of anthropogenic vegetation, but a negative relationship with biodiversity. My method effectively detected human presence hotspots and highlighted the most vulnerable areas. Therefore, multi-criteria decision analysis, used alongside plant ecology expertise, can successfully promote balanced land-use planning, especially in highly frequented regions with significant natural features.

Prioritization of strategic initiatives in the context of natural disaster prevention

Goal: The purpose of this paper is to build the structure of a multicriteria decision model that supports definition and prioritization of strategic initiatives by an institution performing in the prevention of natural disasters.
 Design/Methodology/Approach: The Multicriteria Decision Analysis (MCDA) methodological process is adopted to build a multicriteria evaluation model. A two-phase process is followed, employing a top-down approach, based on Value-Focused Thinking (VFT), in combination with the Multi Attribute Value Theory (MAVT) method. The participants of the MCDA process were experts of the control room in the case study institution.
 Results: The proposed methodology not only helped decision makers to enumerate a number of strategic initiatives to accomplish the organizational objective, but also helped them to establish a structured procedure to prioritize these initiatives.
 Limitations: Absence of a criterion related to the organization budget in the model; limited scope of participation in the process; absence of quantitative criteria.
 Practical implications: The major practical contributions are as follows: an structured model that supports the strategic planning process; a better allocation of resources (human, financial, and materials) in projects that are truly aligned with the strategic objective of the organization; the organizational learning coming from the exercise of reflection on values, objectives and preferences; and the legitimacy of decisions as a result of the participative character of the construction process of the model.
 Originality/Value: In this study, a multicriteria evaluation model is structured and applied as support for strategic decision making in the context of a natural disaster early warning system. The model has a significant application potential, since it encourages the adoption of structured decision support methods rather than traditional empirical decision making. Thus, the value of the study lies in the contribution that the proposed model can offer to more effective disaster prevention.

Participatory multi-criteria decision analysis in valuing peatland ecosystem services—Trade-offs related to peat extraction vs. pristine peatlands in Southern Finland

The use of peatlands and the multiple but mutually exclusive ecosystem services they provide is a highly debated issue worldwide. We used a participatory multi-criteria decision analysis (MCDA) to address multiple value dimensions and trade-offs related to peatland ecosystem services in Southern Finland. We evaluated five peatland policy scenarios against provisioning, regulating and cultural ecosystem services as well as socio-economic factors, and engaged key stakeholders in framing the assessment and assigning criteria weights. The MCDA process showed that while peat extraction can basically be reconciled with preserving the most important biodiversity values in Finland, the conflict between peat extraction and carbon stock as well as water quality impacts and the related amenity values is irreconcilable. The role of the participatory MCDA process in promoting learning and reflection was smaller than expected but it did facilitate learning about the flows of peatland ecosystem services. The role of the participants was important not only in making value judgements but also in contributing to the impact assessment, thereby supporting the calls for transdisciplinarity in ecosystem service assessments.

Methods to inform the development of concise objectives hierarchies in multi-criteria decision analysis

Building a well-structured objectives hierarchy is central to multi-criteria decision analysis (MCDA). However, in the absence of a systematic methodology to support the process, this task has been described as “more art than science”. Objectives hierarchies often tend to become large and constraining the size of a hierarchy can be challenging. This paper proposes and illustrates the use of a set of methods to support the simplification of the hierarchies in contexts that are “data rich” and characterised by many objectives. The aim of using the proposed approach is to support decision analysts in developing an appropriately concise decision model for the further interactions with the stakeholders. Using data from two completed environmental cases we show retrospectively how qualitative (means-ends networks), semi-quantitative (relevancy analysis) and quantitative (correlation analysis, principal component analysis, local sensitivity analysis of weights) methods, used alone or in combination, can inform hierarchy development. We evaluate the potential benefits and challenges of each method and discuss the advantages and disadvantages of the simplification of an objectives hierarchy. Questionnaire-based relevancy analysis can be a useful method to identify and communicate important objectives in the early phases of an MCDA process with stakeholders, while correlation analysis can help to identify overlapping objectives, particularly in cases having many objectives and alternatives. It is intended that the methods support a facilitator in developing a clear understanding of the problem and also prompt deeper thinking about and discussion of the appropriate structure and content of an objectives hierarchy with the stakeholders involved.

Multicriteria decision analysis process to develop new classification criteria for systemic lupus erythematosus

European League Against Rheumatism and are jointly supporting multiphase development of systemic lupus erythematosus (SLE) classification criteria based on weighted criteria and a continuous probability scale. Prior steps included item...

Healthcare decision‐making applications using multicriteria decision analysis: A scoping review

AbstractThe objective of this study is to review applications of multicriteria decision analysis (MCDA) methods to provide structure for and practical insights on how MCDA methods are applied in different healthcare areas. Four databases (Embase, Medline, Web of Science, and PubMed) were searched from 1980 to February 7, 2018. A scoping review was performed to outline the most frequently applied MCDA methods, in which only case studies were considered, as the objective was to assess the applications of MCDA methods. Seventy case studies were selected. The review showed that the first real‐life application was published in 1990. With regard to areas of application, three aspects were analysed: type of health services (public or private), type of intervention (preventive, diagnostic, or treatment), and healthcare area (policymaking; resource allocation; health technology assessment; pharmaceutics; and hospital, clinical, or patient level). Data revealed that MCDA was principally used in public health services (n = 59). Three types of interventions were identified: prevention (n = 36), diagnosis (n = 32), and treatment (n = 2). An MCDA process of application was then followed to provide practical insights. During the problem‐structuring phase, results showed that processes for selecting stakeholders and criteria differed among healthcare areas. For model building, authors preferred value measurement methods and especially the analytical hierarchy process. Authors of the 70 case studies included and confirmed the usefulness of MCDA in different healthcare contexts. The stakeholders, criteria, and MCDA methods chosen differ from one study to another, answering different research objectives in different healthcare areas. This scoping review highlights the importance of each MCDA step (using Belton and Stewart's framework), from problem structuring (including the selection of stakeholders) to the model building phase (encompassing the selection of the appropriate MCDA method). We recommend structured and justified analysis so that decision makers and stakeholders can feel confident during the application process in order to make final decisions more meaningful. Further work is needed to help researchers and decision makers when choosing a suitable MCDA method.

Use of Analyst-Generated Stakeholder Preference Profiles in Multi-Criteria Decision Analysis — Experiences from an Urban Planning Case

Multi-criteria decision analysis (MCDA) is a generic term for approaches supporting the systematic evaluation of alternatives in problems involving multiple criteria and stakeholders. One of the most challenging tasks is to gather preference information from stakeholders in a way that both reflects their true opinions and meets the theoretical requirements of the applied MCDA approach. Various techniques have been used in practice, including interviews and decision conferences. In this paper, we present a new cost-efficient approach in which an analyst generates weight profiles for various stakeholder groups. That is, instead of personally asking specific trade-off questions from the stakeholders, weight profiles are developed on the basis of more general preference information collected from the stakeholders. The potential advantages of this approach are: (i) the collection of the preferences using surveys is less laborious than personal interviews or decision conferences and (ii) the risk of cognitive biases in the weight elicitation can be reduced, because the most challenging task of MCDA — assigning weights to the criteria — is left to the analyst, who should be aware of typical biases and how to avoid them. We developed and tested the approach in a contested public decision-making situation related to the development of a new residential area. We utilised the data gathered from the participants of the workshops (21) as well as the data from a web survey including 177 responds via a randomly sampled closed survey, in addition to 484 responds via an open survey. Four preference profiles each having specific weight distributions to criteria were developed, using a multi-stage procedure. Four development alternatives were compared as based on the developed preference profiles. We were able to realise the MCDA process within a very tight time schedule, create plausible preference profiles and summarise each alternative’s pros and cons from different perspectives. However, we also identified several issues which have to be paid more attention in future cases or require further research.

A new multi-criteria decision process to prioritise requirements

Most software projects have more candidate requirements than can be realised within the time and cost constraints. This paper presents a novel multi-criteria decision analysis process to prioritise requirements. The novelty of the presented idea is three-fold. Firstly, to prioritise requirements, it distinguishes two categories of requirements according to their level of abstraction: low-level requirements and high-level requirements. Then, it relates business goals to the requirements of low-level which contribute to their fulfilment. This will improve the completeness and traceability of requirements. Secondly, requirements prioritisation is based on their degree of contribution to the identified business goals and their importance. So, business goals become the evaluation criteria. Finally, this process takes into account relationships and dependencies that may exist between business goals. At this end, we employ analytic hierarchy process (AHP) method to give weights to business goals and Choquet integral to calculate a global score, i.e. priority, for requirements.

Applying Multi-Criteria Decision Analysis (MCDA) Simple Scoring as an Evidence-based HTA Methodology for Evaluating Off-Patent Pharmaceuticals (OPPs) in Emerging Markets

Off-patent pharmaceuticals (OPPs) represent more than 60% of the pharmaceutical market in many emerging countries, where they are frequently evaluated primarily on cost rather than with health technology assessment. OPPs are assumed to be identical to the originators. Branded and unbranded generic versions can, however, vary from the originator in active pharmaceutical ingredients, dosage, consistency formulation, excipients, manufacturing processes, and distribution, for example. These variables can alter the efficacy and safety of the product, negatively impacting both the anticipated cost savings and the population’s health. In addition, many health care systems lack the resources or expertise to evaluate such products, and current assessment methods can be complex and difficult to adapt to a health system’s needs. Multicriteria decision analysis (MCDA) simple scoring is an evidence-based health technology assessment methodology for evaluating OPPs, especially in emerging countries in which resources are limited but decision makers still must balance affordability with factors such as drug safety, level interchangeability, manufacturing site and active pharmaceutical ingredient quality, supply track record, and real-life outcomes. MCDA simple scoring can be applied to pharmaceutical pricing, reimbursement, formulary listing, and drug procurement. In November 2015, a workshop was held at the International Society for Pharmacoeconomics and Outcomes Research Annual Meeting in Milan to refine and prioritize criteria that can be used in MCDA simple scoring for OPPs, resulting in an example MCDA process and 22 prioritized criteria that health care systems in emerging countries can easily adapt to their own decision-making processes.

Discrepancies between multicriteria decision analysis-based ranking and intuitive ranking for pharmaceutical benefit-risk profiles in a hypothetical setting.

Multicriteria decision analysis (MCDA) has been generally considered a promising decision-making methodology for the assessment of drug benefit-risk profiles. There have been many discussions in both public and private sectors on its feasibility and applicability, but it has not been employed in official decision-makings. For the purpose of examining to what extent MCDA would reflect the first-hand, intuitive preference of evaluators in practical pharmaceutical assessments, we conducted a questionnaire survey involving the participation of employees of pharmaceutical companies. Showing profiles of the efficacy and safety of four hypothetical drugs, each respondent was asked to rank them following the standard MCDA process and then to rank them intuitively (i.e. without applying any analytical framework). These two approaches resulted in substantially different ranking patterns from the same individuals, and the concordance rate was surprisingly low (17%). Although many respondents intuitively showed a preference for mild, balanced risk-benefit profiles over profiles with a conspicuous advantage in either risk or benefit, the ranking orders based on MCDA scores did not reflect the intuitive preference. Observed discrepancies between the rankings seemed to be primarily attributed to the structural characteristics of MCDA, which assumes that evaluation on each benefit and risk component should have monotonic impact on final scores. It would be difficult for MCDA to reflect commonly observed non-monotonic preferences for risk and benefit profiles. Possible drawbacks of MCDA should be further investigated prior to the real-world application of its benefit-risk assessment.