Uncertainty In Ranking Research Articles

Cluster Analysis Methods to Support Population Health Improvement Among US Counties.

Population health rankings can be a catalyst for the improvement of health by drawing attention to areas in need of relative improvement and summarizing complex information in a manner understood by almost everyone. However, ranks also have unintended consequences, such as being interpreted as "hard truths," where variations may not be significant. There is a need to improve communication about uncertainty in ranks, with accurate interpretation. The most common solutions discussed in the literature have included modeling approaches to minimize statistical noise or borrow strength from covariates. However, the use of complex models can limit communication and implementation, especially for broad audiences. Explore data-informed grouping (cluster analysis) as an easier-to-understand, empirical technique to account for rank imprecision that can be effectively communicated both numerically and visually. Cluster analysis, specifically k-means clustering with Wasserstein (earth mover's) distance, was explored as an approach to identify natural and meaningful groupings and gaps in the data distribution for the County Health Rankings' (CHR) health outcomes ranks. County-level health outcomes from the 2022 CHR. 3082 counties that were ranked in the 2022 CHR. Data-informed health groups. Cluster analysis identified 30 health groupings among counties nationwide, with cluster size ranging from 9 to 184 counties. On average, states had 16 identified clusters, ranging from 3 in Delaware and Hawaii to 27 in Virginia. Number of clusters per state was associated with number of counties per state and population of the state. The method helped address many of the issues that arise from providing rank estimates alone. Public health practitioners can use this information to understand uncertainty in ranks, visualize distances between county ranks, have context around which counties are not meaningfully different from one another, and compare county performance to peer counties.

Assessment, quantification and propagation of uncertainty in seismic life-cycle cost analysis

Seismic life-cycle cost analysis (SLCCA) is a comprehensive tool to estimate the expected lifetime economic losses due to seismic events in terms of present-day value and aids in lifetime investment planning and design decision-making. The state-of-the-art framework for SLCCA incorporates three essential components: seismic hazard analysis, seismic fragility curves and damage cost estimates. Due to different sources of uncertainty prevailing in each component, the SLCC estimates may vary substantially. However, most of the existing literature primarily focuses on the expected value of the SLCC, and uncertainties stemming from multiple sources are unaccounted for. To address this drawback, this study proposes a comprehensive approach to quantify and propagate multiple uncertainties and offers a visualization of the shape and nature of SLCC distribution. Furthermore, the application of the framework is demonstrated using a case study example of the reinforced concrete (RC) frame building. Results reveal that the SLCC distribution is multimodal due to multiple uncertainty sources and the expected SLCC values also shift significantly compared to the state-of-the-art approaches, highlighting the criticality of the uncertainty propagation. Lastly, a novel framework is proposed to rank uncertainty sources based on their relative influence that will aid stakeholders in informed decision-making under uncertainty.

Ablative and non-surgical therapies for early and very early hepatocellular carcinoma: a systematic review and network meta-analysis.

A wide range of ablative and non-surgical therapies are available for treating small hepatocellular carcinoma in patients with very early or early-stage disease and preserved liver function. To review and compare the effectiveness of all current ablative and non-surgical therapies for patients with small hepatocellular carcinoma (≤ 3 cm). Systematic review and network meta-analysis. Nine databases (March 2021), two trial registries (April 2021) and reference lists of relevant systematic reviews. Eligible studies were randomised controlled trials of ablative and non-surgical therapies, versus any comparator, for small hepatocellular carcinoma. Randomised controlled trials were quality assessed using the Cochrane Risk of Bias 2 tool and mapped. The comparative effectiveness of therapies was assessed using network meta-analysis. A threshold analysis was used to identify which comparisons were sensitive to potential changes in the evidence. Where comparisons based on randomised controlled trial evidence were not robust or no randomised controlled trials were identified, a targeted systematic review of non-randomised, prospective comparative studies provided additional data for repeat network meta-analysis and threshold analysis. The feasibility of undertaking economic modelling was explored. A workshop with patients and clinicians was held to discuss the findings and identify key priorities for future research. Thirty-seven randomised controlled trials (with over 3700 relevant patients) were included in the review. The majority were conducted in China or Japan and most had a high risk of bias or some risk of bias concerns. The results of the network meta-analysis were uncertain for most comparisons. There was evidence that percutaneous ethanol injection is inferior to radiofrequency ablation for overall survival (hazard ratio 1.45, 95% credible interval 1.16 to 1.82), progression-free survival (hazard ratio 1.36, 95% credible interval 1.11 to 1.67), overall recurrence (relative risk 1.19, 95% credible interval 1.02 to 1.39) and local recurrence (relative risk 1.80, 95% credible interval 1.19 to 2.71). Percutaneous acid injection was also inferior to radiofrequency ablation for progression-free survival (hazard ratio 1.63, 95% credible interval 1.05 to 2.51). Threshold analysis showed that further evidence could plausibly change the result for some comparisons. Fourteen eligible non-randomised studies were identified (n ≥ 2316); twelve had a high risk of bias so were not included in updated network meta-analyses. Additional non-randomised data, made available by a clinical advisor, were also included (n = 303). There remained a high level of uncertainty in treatment rankings after the network meta-analyses were updated. However, the updated analyses suggested that microwave ablation and resection are superior to percutaneous ethanol injection and percutaneous acid injection for some outcomes. Further research on stereotactic ablative radiotherapy was recommended at the workshop, although it is only appropriate for certain patient subgroups, limiting opportunities for adequately powered trials. Many studies were small and of poor quality. No comparative studies were found for some therapies. The existing evidence base has limitations; the uptake of specific ablative therapies in the United Kingdom appears to be based more on technological advancements and ease of use than strong evidence of clinical effectiveness. However, there is evidence that percutaneous ethanol injection and percutaneous acid injection are inferior to radiofrequency ablation, microwave ablation and resection. PROSPERO CRD42020221357. This award was funded by the National Institute for Health and Care Research (NIHR) Health Technology Assessment (HTA) programme (NIHR award ref: NIHR131224) and is published in full in Health Technology Assessment; Vol. 27, No. 29. See the NIHR Funding and Awards website for further award information.

Inherent safety of clean fuels for maritime transport

The urgent need to reduce the emission of harmful pollutants in maritime transport promoted the development of several alternative propulsion systems based on clean fuels or carbon-neutral energy vectors. However, the alternative solutions under development pose new concerns from the safety perspective. Thus, an innovative methodology to rank the inherent safety performance of alternative systems at early design stages was developed. A case study representative of long-distance maritime transportation was analysed. The inherent safety performances of Liquefied Natural Gas (LNG), Liquid Hydrogen (LH2), and Liquid Ammonia (LNH3) were compared to that of Marine Gas Oil (MGO), assumed as a benchmark representing state-of-the-art technologies. Uncertainty and robustness of the safety ranking obtained were tested via a Monte Carlo analysis. The results show that technologies based on LNG have similar safety performances with respect to the benchmark option. Conversely, LH2 safety performance is currently limited by the lack of mature technologies for its safe storage whilst the safety of LNH3-based applications is affected by the toxicity of ammonia.

Pharmacological and nonpharmacological augmentation treatments for clozapine-resistant schizophrenia: A systematic review and network meta-analysis with normalized entropy assessment.

To integrate all evidence derived from randomized controlled trials (RCTs) of both pharmacological and nonpharmacological augmentation interventions for clozapine-resistant schizophrenia (CRS). Six major electronic databases were systematically searched for RCTs published until July 10, 2021. The primary outcome was change in overall symptoms, and the secondary outcomes were positive and negative symptoms and acceptability. We performed random-effects network meta-analysis. Normalized entropy was calculated to examine the uncertainty of treatment ranking. We identified 35 RCTs (1472 patients with 23 active augmentation treatments) with a mean daily clozapine dose of 440.80 (91.27) mg for 1168.22 (710.28) days. Network meta-analysis of overall symptoms (reported as standardized mean difference; 95 % confidence interval) with consistent results indicated that mirtazapine (-4.41; -5.61, -3.21), electroconvulsive therapy (ECT) (-4.32; -5.43, -3.21), and memantine (-2.02; -3.14, -0.91) were ranked as the best three treatments. For positive symptoms, ECT (-5.18; -5.86, -4.49) was ranked the best with less uncertainty. For negative symptoms, memantine (-3.38; -4.50, -2.26), duloxetine (-3.27; -4.25, -2.29), and mirtazapine (-1.73; -2.71, -0.74) were ranked the best three treatments with less uncertainty. All antipsychotics, N-methyl d-aspartate receptor agonists, and antiepileptics were not associated with more efficacy than placebo. Compared to placebo, only amisulpride had statistically significant lower discontinuation rate (risk ratio: 0.21; 95 % CI: 0.05, 0.93). Add-on mirtazapine, ECT, and memantine were the most efficacious augmentation options for CRS. Data on other important outcomes such as cognitive functioning or quality of life were rarely reported, making further large-scale, well-designed RCTs necessary. (PROSPERO number, CRD42021262197.).

Uncertainty in Law School Citation Rankings

Uncertainty in Law School Citation Rankings

QU-BraTS: MICCAI BraTS 2020 Challenge on Quantifying Uncertainty in Brain Tumor Segmentation – Analysis of Ranking Scores and Benchmarking Results

Deep learning (DL) models have provided the state-of-the-art performance in a wide variety of medical imaging benchmarking challenges, including the Brain Tumor Segmentation (BraTS) challenges. However, the task of focal pathology multi-compartment segmentation (e.g., tumor and lesion sub-regions) is particularly challenging, and potential errors hinder the translation of DL models into clinical workflows. Quantifying the reliability of DL model predictions in the form of uncertainties, could enable clinical review of the most uncertain regions, thereby building trust and paving the way towards clinical translation. Recently, a number of uncertainty estimation methods have been introduced for DL medical image segmentation tasks. Developing scores to evaluate and compare the performance of uncertainty measures will assist the end-user in making more informed decisions. In this study, we explore and evaluate a score developed during the BraTS 2019-2020 task on uncertainty quantification (QU-BraTS), and designed to assess and rank uncertainty estimates for brain tumor multi-compartment segmentation. This score (1) rewards uncertainty estimates that produce high confidence in correct assertions, and those that assign low confidence levels at incorrect assertions, and (2) penalizes uncertainty measures that lead to a higher percentages of under-confident correct assertions. We further benchmark the segmentation uncertainties generated by 14 independent participating teams of QU-BraTS 2020, all of which also participated in the main BraTS segmentation task. Overall, our findings confirm the importance and complementary value that uncertainty estimates provide to segmentation algorithms, and hence highlight the need for uncertainty quantification in medical image analyses. Our evaluation code is made publicly available at <a href='https://github.com/RagMeh11/QU-BraTS'>https://github.com/RagMeh11/QU-BraTS</a>

High robustness does not always imply low uncertainty of treatment rankings: An empirical study of 60 network meta-analyses

Background: Network meta-analysis computes treatment ranking to assist with clinical decision making, but it is not always clear how reliable the ranking is and how likely the accumulation of new evidence may alter the ranking. Uncertainty and robustness of ranking are two concepts related to the reliability of ranking. However, it is still unclear whether these two approaches would always yield similar conclusions on the reliability of ranking, i.e., a robust ranking is also one of low uncertainty. Purpose: This study aimed to investigate the relationship between the uncertainty and robustness of treatment ranking by using normalized entropy and quadratic weighted Cohen’s kappa, respectively. Data. We used datasets of previously published NMAs from a database maintained by Petropoulou et al. at the University of Bern. Analysis. Scatter plots and Pearson’s correlation coefficients were used to demonstrate the direction and strength of the association between uncertainty and robustness of ranking for NMA-level and treatment-level evaluation. Results: We found that when the uncertainty of ranking is very low, treatment ranking is unlikely to be altered by deleting a trial from the complete data. However, network meta-analysis with robust treatment ranking may have high uncertainty of treatment ranking. Conclusions: Therefore, although the robustness of the ranking can find the trial that has the most significant impact on the ranking, the high robustness of ranking does not mean that the ranking would not easily change when new trials are added in the future.

A novel IVIF QFD considering both the correlations of customer requirements and the ranking uncertainty of technical attributes

A novel IVIF QFD considering both the correlations of customer requirements and the ranking uncertainty of technical attributes

Measuring the spatial distribution of health rankings in the United States.

We rank counties in the United States with respect to population health. We utilize the five observable county health variables used to construct the University of Wisconsin Population Health Institute's County Health Rankings (CHRs). Our method relies on a Bayesian factor analysis model that estimates data-driven weights for our rankings, incorporates county population sizes into the level of rank uncertainty, and allows for spillovers of health stock across county lines. We find that demographic and economic variation explains a large portion of the variation in health rankings. We address the importance of uncertainty caused by imputation of missing data and show that there is a substantial quantity of uncertainty in rankings throughout the rank distribution. Analyzing the health of counties both within and across state lines shows notable degrees of disparity in county health. While we find some disagreement between the ranks of our model and the CHRs, we show that there is additional information gained by utilizing the rankings produced by both methods.

Using Normalized Entropy to Measure Uncertainty of Rankings for Network Meta-analyses.

Ranking of treatments offers a straightforward interpretation of results derived from network meta-analysis. However, some published network meta-analyses have overemphasized treatment ranking without paying attention to its uncertainty. According to a review of 91 network meta-analyses, 52 reported treatment ranking, but 43 of them did not report the uncertainty of ranking. Without reporting the uncertainty, small differences in the ranking of treatments may be overinterpreted. Rankograms, cumulative rankograms, the credible/confidence interval of mean rank, the surface under the cumulative ranking curve (SUCRA), and the interquartile range of median rank have been used to show the uncertainty of rankings. However, it is not always straightforward to compare the differences in the distribution of probabilities by inspecting rankograms or to compare the intervals or ranges of treatment ranks. We therefore proposed normalized entropy, which transforms the distribution of ranking probabilities into a single quantitative measure, to facilitate a refined interpretation of uncertainty of treatment ranking. We used 4 real examples to demonstrate the uncertainty of ranking quantified by ranking probabilities, 95% confidence interval of SUCRA, and normalized entropy. We showed that as normalized entropy ranges from 0 to 1 and is independent of the number of treatments, it can be used to compare the uncertainty of treatment ranking within a network meta-analysis (NMA) and between different NMAs. Normalized entropy is an alternative tool for measuring the uncertainty of treatment ranking by improving the translation of results from NMAs to clinical practice and avoiding naïve interpretation of treatment ranking. We therefore recommend normalized entropy to be included in the presentation and interpretation of results from NMAs.

Rank Uncertainty in Organizations

A principal incentivizes a team of agents to work by privately offering them bonuses contingent on team success. We study the principal’s optimal incentive scheme that implements work as a unique equilibrium. This scheme leverages rank uncertainty to address strategic uncertainty. Each agent is informed only of a ranking distribution and his own bonus, the latter making work dominant provided that higher-rank agents work. If agents are symmetric, their bonuses are identical. Thus, discrimination is strictly suboptimal, in sharp contrast with the case of public contracts (Winter 2004). We characterize how agents’ ranking and compensation vary with asymmetric effort costs. (JEL D23, D62, D81, D82, D86)

Interspecific social dominance networks reveal mechanisms promoting coexistence in sympatric charr in Hokkaido, Japan.

Coexistence of species requires equalizing mechanisms that minimize fitness differences, which are balanced by stabilizing mechanisms that enhance negative intraspecific interactions versus interspecific ones. Here, we develop a simple theoretical framework that allows measuring the relative strength of intraspecific versus interspecific competition in dominance hierarchies. We use it to evaluate mechanisms promoting coexistence between two congeneric charr that compete for foraging positions, which strongly influence density-dependent growth and survival. Agonistic interactions (n=761) among 71 Dolly Varden Salvelinus malma and whitespotted charr Salvelinus leucomaenis were measured by snorkelling in two pools in the sympatric zone of a Hokkaido stream during two summers. Interspecific dominance hierarchies, analysed using three methods, were closely correlated with fish length but the species treated each other equally. Ranks for the most dominant fish in each pool, determined directly by knockout experiments, were also virtually identical to ranks by length. Similarly, exponential random graph modelling of the social networks provided no evidence that either species was dominant over the other. Instead, larger fish were more likely to win contests, especially over fish of the next lower ranks. These results demonstrated that the two species were nearly ecological equivalents in accessing key resources in this sympatric zone. Nearly identical growth and stable densities over 4years further supported this inference, although Dolly Varden were a minority (29% of the assemblage), a sign of some fitness difference. Detailed foraging observations coupled with two concurrent studies revealed an effective stabilizing mechanism. Dolly Varden shifted to feeding directly from the benthos when drifting invertebrates declined, a behaviour enhanced by morphological character displacement, thereby partitioning food resources and enhancing intraspecific competition while avoiding agonistic encounters with whitespotted charr. The plurality of evidence indicates that fitness differences between these ecologically equivalent species are small in this local assemblage, and balanced by resource partitioning, a modest stabilizing mechanism that promotes coexistence. The theoretical framework presented here is a useful tool to evaluate the strength of interspecific versus intraspecific competition, which combined with information on trade-offs in ecological performance can contribute to a mechanistic understanding of species coexistence.

Cost-effectiveness of antithrombotic agents for atrial fibrillation in older adults at risk for falls: a mathematical modelling study.

Antithrombotic drugs decrease stroke risk in patients with atrial fibrillation, but they increase bleeding risk, particularly in older adults at high risk for falls. We aimed to determine the most cost-effective antithrombotic therapy in older adults with atrial fibrillation who are at high risk for falls. We conducted a mathematical modelling study from July 2019 to March 2020 based on the Ontario, Canada, health care system. We derived the base-case age, sex and fall risk distribution from a published cohort of older adults at risk for falls, and the bleeding and stroke risk parameters from an atrial fibrillation trial population. Using a probabilistic microsimulation Markov decision model, we calculated quality-adjusted life years (QALYs), total cost and incremental cost-effectiveness ratios (ICERs) for each of acetylsalicylic acid (ASA), warfarin, apixaban, dabigatran, rivaroxaban and edoxaban. Cost data were adjusted for inflation to 2018 values. The analysis used the Ontario public payer perspective with a lifetime horizon. In our model, the most cost-effective antithrombotic therapy for atrial fibrillation in older patients at risk for falls was apixaban, with an ICER of $8517 per QALY gained (5.86 QALYs at $92 056) over ASA. It was a dominant strategy over warfarin and the other antithrombotic agents. There was moderate uncertainty in cost-effectiveness ranking, with apixaban as the preferred choice in 66% of model iterations (given willingness to pay of $50 000 per QALY gained); edoxaban, 30 mg, was preferred in 31% of iterations. Sensitivity analysis across ranges of age, bleeding risk and fall risk still favoured apixaban over the other medications. From a public payer perspective, apixaban is the most cost-effective antithrombotic agent in older adults at high risk for falls. Health care funders should implement strategies to encourage use of the most cost-effective medication in this population.

Probabilistic performance estimators for computational chemistry methods: Systematic improvement probability and ranking probability matrix. II. Applications.

In Paper I [P. Pernot and A. Savin, J. Chem. Phys. 152, 164108 (2020)], we introduced the systematic improvement probability as a tool to assess the level of improvement on absolute errors to be expected when switching between two computational chemistry methods. We also developed two indicators based on robust statistics to address the uncertainty of ranking in computational chemistry benchmarks: Pinv, the inversion probability between two values of a statistic, and Pr, the ranking probability matrix. In this second part, these indicators are applied to nine data sets extracted from the recent benchmarking literature. We also illustrate how the correlation between the error sets might contain useful information on the benchmark dataset quality, notably when experimental data are used as reference.

Rank Uncertainty in Organizations

A principal incentivizes a team of agents to work by privately offering them bonuses contingent on team success. We study the principal’s optimal incentive scheme that implements work as a unique equilibrium. This scheme leverages rank uncertainty to address strategic uncertainty. Each agent is informed only of a ranking distribution and his own bonus, the latter making work dominant provided that higher-rank agents work. If agents are symmetric, their bonuses are identical. Thus, discrimination is strictly suboptimal, in sharp contrast with the case of public contracts (Winter, 2004). We characterize how agents’ ranking and compensation vary with asymmetric effort costs.

Ranking uncertainty: Wave climate variability versus model uncertainty in probabilistic assessment of coastline change

Sand nourishments are increasingly applied as adaptive coastal protection measures. Predictions of the evolution of these nourishments and their impact on the surrounding coastline contain many uncertainties. The sources that add to this uncertainty can be delineated between intrinsic and epistemic uncertainty, i.e. inevitably in the system or related to knowledge limitations. Effects of intrinsic uncertainty (e.g. due to wave climate variability) on coastal evolution can be significant. In studying these effects, it has often been assumed that intrinsic uncertainty is dominant over epistemic uncertainty (e.g. introduced by the model), yet the magnitude of both contributions have not been explicitly quantified to assess the validity of this assumption. This paper examines the relative importance of intrinsic and epistemic uncertainty in coastline modeling of a large-scale nourishment. It uses a probabilistic framework in which sediment transport is considered to be a function of random wave forcing (intrinsic) and model (epistemic) uncertainty, calculating transport using a one-line model. The test case for this analysis is the mega-nourishment, the Sand Engine, located in the Netherlands. The applied wave climate variability is obtained from long term wave observations, whereas model uncertainty is quantified using the Generalized Likelihood Uncertainty Estimation (GLUE) method relying on monthly observations. We find that the confidence intervals on predicted volume losses increase substantially when including both intrinsic and epistemic sources of uncertainty. A global sensitivity analysis shows that ignoring model uncertainty would underestimate the variance by at least 50% after a 2.5-year simulation period for the Sand Engine, hence producing significant overconfidence in the results. These findings imply that for coastal modeling purposes a dual approach should be considered, evaluating both epistemic and intrinsic uncertainties.

Identifying best performing hospitals in colorectal cancer care; is it possible?

BackgroundComparing outcomes across hospitals to learn from best performing hospitals can be valuable. However, reliably identifying best performance is challenging. This study assesses the possibility to distinguish best performing hospitals on single outcomes and consistency of performance on different outcomes. MethodsData were derived from the Dutch ColoRectal Audit 2013–2015. Outcomes considered were textbook outcome (colon), (circumferential) resection margins, (serious) complications, mortality, and ‘failure to rescue’. To include uncertainty in rankings, random effect logistic regression models were used to calculate expected ranks (ERs), for each hospital and outcome. Rankability was calculated for each outcome, as a measure of reliability of ranking. Furthermore, correlation between ERs on different outcomes was assessed. Correlation was considered weak <0.40, moderate between 0.40 - 0.59 and strong >0.60. ResultsThe study included 32 143 patients; of whom 11 373 were treated in 2015 across 84 hospitals, 8181 colon and 3192 rectal cancer patients. In this one-year period ‘Postoperative complications’ had the highest rankability for colon (57%) and rectal (41%) surgery. No (group of) hospital(s) had the highest ER(s) on all outcomes. Correlation between ERs of outcomes was moderate in 2 (of 25) and strong in 4 (of 25) combinations. Rankability of colorectal mortality increased from 14% in 2015 to 35% when data over 2013–2015 were used. ConclusionThe highest reliability of identifying best performance based on an outcome was 57%. However, the balance between reliability and relevance of outcomes is vulnerable. No (group of) hospital(s) could be identified as best performer on all outcomes. Performance was not consistent on outcomes.

The cloud model based stochastic multi-criteria decision making technology for river health assessment under multiple uncertainties

The river health assessment (RHA) usually concerns multiple criteria including streamflow, ecology, physical structure, water quality and social service functions and evaluation standards formulated by managers as well as groups with different interests. Conventional multi-criteria decision making (MCDM) models are used to comprehensively evaluate river condition through multiple criteria under deterministic environment. In fact, uncertainties are always existed in these criteria data since they are correlated with hydrology and water resources system. In this work, we proposed a stochastic cloud model based MCDM framework for solving RHA considering multiple uncertainties in criteria performance values (PVs) and criteria weights (CWs). The cloud model is allied to describe uncertainty of PVs using numerous drops following normal distribution generated by forward drop generator (FDG). The minimum deviation principle based aggregated CWs are utilized to efficiently quantify uncertainty in CWs and reduce conflict between multiple CWs sources. A novel stochastic multi-criteria acceptability analysis (SMAA) is developed coupling with grey correlation analysis (GCA) and TOPSIS. The risk information for river evaluation caused by multiple uncertainties are described using quantified decision error risk (QDER) and rank uncertainty degree (RUD). The proposed methodology is verified practicability by applying it to river health evaluation in Taihu basin. The numerical simulations are conducted to demonstrate superiority and efficiency of novel SMAA in comparison with conventional SMAA and deterministic MCDM models based on GCA and TOPSIS. The robustness analysis is implemented to disclose its computation stability and reliability as well as effects of cloud parameters on final MCDM results. The results of novel SMAA show that it provides river managers with comprehensive river health and risk analysis information, assisting them to make highly reliable assessment and adopt effective measures harnessing rivers.

A stochastic multi-criteria decision making framework for robust water resources management under uncertainty

Water planner and managers always work in a changing and uncertain environment, since the majority of water resource-related data is imprecise or uncertain to some extent. MCDM (multi-criteria decision making) is a well-suited decision support tool for water resources management, however multiple uncertainties existed in criteria weights (CWs) and performance values (PVs) hinder the application of MCDM methods in real-world problems. This paper proposes an application-oriented stochastic MCDM framework for robust water resources management under uncertainty. A weight aggregation approach based on the game theory coupled with feasible weight space (FWS) is developed for conflict resolution and uncertainty estimation. We propose a novel SMAA-GRA model by combining the stochastic multi-criteria acceptability analysis (SMAA) theory and grey relational analysis (GRA). The risk of errors in decision making and rank uncertainty degree are utilized to assess decision-making uncertainty. We perform significance analysis to examine the impacts of input parameter uncertainties on the final assessment of alternatives. The proposed framework is demonstrated via two water resources management case studies. Numerical experiments are conducted to investigate the robustness and computational efficiency of the framework as well as the effects of uncertainty level and probability distribution type on decision-making results. The results indicate that the framework enables water managers to examine the robustness of final decisions and allows a risk-informed water resource decision to be made with higher reliabilities.