Expert Uncertainty Research Articles

Weibull fuzzy probability distribution for analysing the behaviour of pulping unit in a paper industry

The purpose of this paper is to present a technique for analysing the behaviour of an industrial system stochastically by utilising vague, imprecise, and uncertain data. The technique utilises Petri nets and fuzzy Lambda-Tau method for analysing the reliability indices of time varying failure rate instead of the constant failure rate. Petri nets are used for modelling the system while fuzzy set theory is used for representing the failure rate and repair time because fuzzy numbers allows expert opinions, operating conditions, uncertainty and imprecision in reliability information. Various expressions of reliability indices like failure rate, repair time, mean time between failures (MTBF), reliability, availability and maintainability for the system in terms of Weibull distribution is computed. The pulping unit of a paper mill situated in a northern part of India, producing approximately 200 tons of paper per day, has been considered to demonstrate the proposed approach.

Characterizing disagreement and inconsistency in experts' judgments in the analytic hierarchy process

PurposeMethods and techniques of aggregating preferences or priorities in the analytic hierarchy process (AHP) usually ignore variation or dispersion among experts and are vulnerable to extreme values (generated by particular viewpoints or experts trying to distort the final ranking). The purpose of this paper is to propose a modelling approach and a graphical representation to characterize inconsistency and disagreement in the group decision making in the AHP.Design/methodology/approachThe authors apply a regression approach for estimating the decision weights of the AHP using linear mixed models (LMM). They also test the linear mixed model and the multi‐dimensional scaling graphical display using a case of strategic performance management in education.FindingsIn addition to determining the weight vectors, this model also allows the authors to decompose the variation or uncertainty in experts' judgment. Well‐known statistical theories can estimate and rigorously test disagreement among experts, the residual uncertainty due to rounding errors in AHP scale, and the inconsistency within individual experts' judgments. Other than characterizing different sources of uncertainty, this model allows the authors to rigorously test other factors that might significantly affect weight assessments.Originality/valueThis study provides a model to better characterize different sources of uncertainty. This approach can improve decision quality by allowing analysts to view the aggregated judgments in a proper context and pinpoint the uncertain component that significantly affects decisions.

Public Reactions to Risk Messages Communicating Different Sources of Uncertainty: An Experimental Test

ABSTRACT There is an abundant literature on the challenge of integrating uncertainties in experts’ risk assessments, but the evidence on the way they are understood by the public is scarce and mixed. This study aims to better understand the effect of communicating different sources of uncertainty in risk communication. A causal design was employed to test the effect of communicating risk messages varying in type of advisory warning (no risk and suggests no protective measure, or risk and recommends a protective measure) and sources of uncertainty (no uncertainty, divergence between experts, contradictory data, or lack of data) on public reactions. Participants from the general public (N = 434) were randomly assigned to read and react to variants of a fictitious government message discussing the presence of a new micro-organism found in tap water. Multiple analysis of variance showed that to report uncertainty from divergence between experts or from contradictory data reduced the adherence to the message, but not to mention the lack of data. Moreover, the communication of diverse sources of uncertainty did not affect trust in the government when the advisory warning stated there was a risk and recommended a protective measure. These findings have important implications for risk communication.

Risk, expert uncertainty, and Australian news media: public and private faces of expert opinion during the 2009 swine flu pandemic

During the outbreak of emerging infectious diseases scientists and public health officials play a key role in informing communities about what is happening, why and what they can do about it, and the news media are critical to how expert knowledge is presented to the public. Therefore, it is crucial to examine the media-oriented practices of experts and to gauge their perceptions of risk communication during public health emergencies. This study investigates the experiences of scientists and public health officials who were sources for the Australian news media during the 2009 pH1N1 (swine flu) pandemic. The paper discusses the perception among some participants that their colleagues were ‘toeing the party’ line as distinct from offering individual ‘expert opinion’, and identifies the different meanings they associated with responsible expert risk communication. In their encounters with the media some indicated frustration with news media constraints, while others demonstrated an internalization of the kind of skills required to play the ‘media game’. The paper discusses the ways in which scientific experts balanced their roles and responsibilities in the context of institutional pressures to be an active media performer. The study affirms the importance of looking beyond media texts and to the activities and perceptions of media sources in order to fully appreciate framing contests, especially when there is a suggestion that dissenting views on disease risk are restricted from entering public debate.

CLOUD DELPHI METHOD

Group decision making is an important category of problem solving techniques for complicated problems, among which the Delphi method has been widely applied. In this paper an improved Delphi method based on Cloud model is proposed in order to deal with the fuzziness and uncertainty in experts' subjective judgments. The proposed Cloud Delphi Method (CDM) describes experts' opinions by Cloud model and we aggregate the experts' Cloud opinions by synthetic algorithm and weighted average algorithm. Another key point of CDM is to stabilize and accommodate the individual fuzzy estimates by the defined stability rules rather than having to force them to converge, or reduce. The Cloud opinions and aggregation results can be exhibited in a graphically way leading experts to judge intuitively and it can decrease the number of repetitive surveys and/or interviews. Moreover, it is more scientific and easier to represent experts' opinion base on Cloud model which can combine fuzziness and uncertainty well. A numerical example is examined to demonstrate applicability and implementation process of CDM.

Hypothesis Testing with Interval Data: Case of Regulatory Constraints

In many practical situations, there exist regulatory thresholds: e.g., a concentration of certain chemicals in the car exhaust cannot exceed a certain level, etc. In this paper, we describe how to make accept/reject decisions under measurement or expert uncertainty in case of regulatory and expert-based thresholds-where the threshold does not come from a detailed statistical analysis. This paper expands our conference paper [23].

Wise selection of an indicator for monitoring the success of management actions

Objective approaches are needed to select species for monitoring management effectiveness and to justify conservation investments. We develop and apply two approaches for selecting the most cost-effective and informative indicator species with a case study of monitoring investments in invasive predator control in south-western Australia. We compared species selected by a qualitative approach involving experts scoring 17 criteria for 12 potential indicators to a quantitative metric based on empirical data for a subset of criteria. Both approaches accounted for the monitoring cost, potential for leverage, data uncertainty and the risk of choosing an un-representative or un-informative indicator, albeit in different ways. The indicator species selected by the quantitative metric was the western brushtail possum Trichosurus vulpecula – a relatively common medium-sized mammal in the study region. When costs were not considered explicitly in the qualitative approach, the indicator selected was the woylie Bettongia penicillata. When costs were sensibly incorporated in the qualitative approach, indicator rankings changed and the top-ranked species was again T. vulpecula. Although when costs were included the qualitative expert-driven approach delivered the same result as the quantitative metric, it was not robust to minor data adjustments, and incorporating expert uncertainty made it difficult to distinguish the value of potential indicator species from each other. The quantitative metric was comparatively robust to minor data adjustments, and enabled evaluation of data uncertainty in a transparent way, presenting a more objective approach that enables explicit selection of the most informative indicator for monitoring management actions.

A comparison of two data analysis techniques and their applications for modeling the carbon dioxide capture process

Improving the efficiency of the carbon dioxide (CO 2) capture process requires a good understanding of the intricate relationships among parameters involved in the process. The objective of this research is to study the nature of relationships among the key parameters using the approaches of artificial neural network and statistical analysis. Our modeling study used the three-year operational data collected from the amine-based post-combustion CO 2 capture process at the International Test Centre of CO 2 Capture (ITC) located in Regina, Saskatchewan of Canada. The goal of CO 2 capture is to capture and remove CO 2 from industrial gas streams before they are released into the atmosphere. The amine solution is used at ITC for absorbing CO 2 from the industrial flue gas, and then the CO 2 is separated from the amine solution. The amine solution recycles for further CO 2 capture and the CO 2 stream can be stored or used for other industrial purposes. This paper describes the data modeling process using the approaches of: (1) statistical analysis and (2) neural network modeling combined with sensitivity analysis. The results from the two modeling process were compared from the perspectives of predictive accuracy, inclusion of parameters, support for exploration and explication of problem space, modeling uncertainty, and involvement of experts. It was observed that the approach of neural network modeling combined with sensitivity analysis achieved much higher accuracy on predicting CO 2 production rate than the statistical study.

“It's not like I can change my mind later”: Reversibility and decision timing in prostate cancer treatment decision-making

ObjectiveTo explore whether reversibility, decision timing, and uncertainty are relevant to men deciding on treatment for localized prostate cancer (LPC). DesignSecondary qualitative data analysis of unstructured interviews. MethodsContent analysis of previously collected qualitative data (31 individual interviews, 5 focus groups). We identified the frequency of references to reversibility, decision timing, and uncertainty and related sub-themes. ResultsWe identified eight themes: reversibility, timing of decision, number of options, “getting it over with,” “the way I make decisions,” uncertainty among experts, desire for certainty, and probability. Fifteen men mentioned reversibility in individual interviews; 13 mentioned the importance of the timing of their decision. Eleven mentioned the importance of the number of options; twelve “the way I make decisions.” Eleven men mentioned the uncertainty of experts, fourteen the desire to “get it over with,” and six a desire for certainty. ConclusionThis study provides compelling preliminary data suggesting that men consider the reversibility, decision timing, and uncertainty in the prostate cancer treatment decision. Practice implicationThese findings may be helpful in enhancing support for men facing the treatment decision.

Elicitation from Large, Heterogeneous Expert Panels: Using Multiple Uncertainty Measures to Characterize Information Quality for Decision Analysis

Decision analysts are frequently called on to help inform decision makers in cases involving considerable uncertainty. In such situations, expert elicitation of parameter values is frequently used to supplement more conventional research. Expert elicitations typically rely on small panels of experts. However, in cases where the information needed for risk management must draw on a broad range of disciplines or types of professional backgrounds and experience, a larger, more heterogeneous expert panel is needed. In this paper we develop a formal protocol and a suite of uncertainty measures for this work. The protocol uses formal survey methods to take advantage of variation in individual expert uncertainty and heterogeneity among experts as a means of quantifying and comparing sources of uncertainty about parameters of interest. We illustrate the use of this protocol with an expert elicitation on the distribution of foodborne illness in the United States across foods. In the survey, experts are asked to attribute illnesses associated with one of eleven major foodborne pathogens to the consumption of one of eleven categories of food. Results show how the distributions of multiple measures of uncertainty (e.g., agreement of experts and uncertainty in knowledge), made feasible by use of a large panel of experts, can help identify which of several types of risk management actions may be most appropriate.

Update of likelihood-based ground-motion model selection for seismic hazard analysis in western central Europe

Scherbaum et al. [(2004) Bull Seismolo Soc Am 94(6): 2164–2185] proposed a likelihood-based approach to select and rank ground-motion models for seismic hazard analysis in regions of low-seismicity. The results of their analysis were first used within the PEGASOS project [Abrahamson et al. (2002), In Proceedings of the 12 ECEE, London, 2002, Paper no. 633] so far the only application of a probabilistic seismic hazard analysis (PSHA) in Europe which was based on a SSHAC Level 4 procedure [(Budnitz et al. 1997, Recommendations for PSHA: guidance on uncertainty and use of experts. No. NUREG/CR-6372-V1). The outcome of this project have generated considerable discussion (Klügel 2005, Eng Geol 78:285–307, 2005b) Eng Geol 78: 285–307, (2005c) Eng Geol 82: 79–85 Musson et al. (2005) Eng Geol 82(1): 43–55]; Budnitz et al. (2005), Eng Geol 78(3–4): 285–307], a central part of which is related to the issue of ground-motion model selection and ranking. Since at the time of the study by Scherbaum et al. [(2004.) Bull Seismolo Soc Am 94(6): 2164–2185], only records from one earthquake were available for the study area, here we test the stability of their results using more recent data. Increasing the data set from 12 records of one earthquake in Scherbaum et al. [(2004) Bull Seismolo Soc Am 94(6): 2164–2185] to 61 records of 5 earthquakes, which have mainly occurred since the publication of the original study, does not change the set of the three top-ranked ground-motion models [Abrahamson and Silva (1997) Seismolo Res Latt 68(1): 94–127; Lussou et al. (2001) J Earthquake Eng 5(1):13–33; Berge-Thierry et al. (2003) Bull Seismolog Soc Am 95(2): 377–389. Only for the lower-ranked models do we obtain modifications in the ranking order. Furthermore, the records from the Waldkirch earthquake (Dec, 5th, 2004, M w = 4.9) enabled us to develop a new stochastic model parameter set for the application of Campbell’s [(2003) Bull Seismolo Soc Am 93(3): 1012–1033] hybrid empirical model to SW Germany and neighbouring regions.

Problems in the application of the SSHAC probability method for assessing earthquake hazards at Swiss nuclear power plants

From 2000 to 2004 a large scale probabilistic seismic hazard analysis (PEGASOS) was created and performed as a research project, sponsored by the Swiss NPP utilities to improve the assessment methodology for seismic risk in support of the plant-specific seismic PRAs. The project followed the methodology of the SSHAC [ Senior Seismic Hazard Analysis Committee (SSHAC), 1997. Recommendations for Probabilistic Seismic Hazard Analysis: Guidance on Uncertainty and Use of Experts. NU-REG/CR-6372] procedures at its most elaborate way—level 4. Before practical implementation was to occur, a detailed review was performed including validation tests and analysis of uncertainty propagation. This paper presents the main results of the review. The review revealed that current PSHA (Probabilistic Seismic Hazard Analysis) methodology as based on logic trees, in conjunction with the SSHAC procedures, potentially leads to a significant overestimation of the seismic hazard in areas with low seismic activity due to the inherent possibilities of unconstrained accumulation of uncertainties. The preliminary results of the project did not pass any of our logical geological–scientific benchmark tests used in our attempts to perform a validation process of the PEGASOS analysis method. Some of the problems encountered are of generic nature and shall be studied carefully before making the decision of whether or not the Swiss nuclear power industry will adopt the recommended use of SSHAC-procedures as a basis for the evaluation of the seismic hazard for individual nuclear power plant seismic PRA without correction.

Supporting knowledge elicitation and consensus building for dempster-shafer decision models

Many decision models that are based on Dempster-Shafer belief functions involve the elicitation of subjective belief data from a group of experts based on qualitative preferences. Given that a major reason for using a group is the assumption that the combined group judgment is likely superior to individual judgment, then the issues of synthesis and consensus assessment and consensus building become important. In this article we present an integrated structured, noninvasive action learning knowledge elicitation process for eliciting from a group of experts the basic probability assignments that are required by Dempster-Shafer theory (DST)-based expert systems. This process accommodates the expert's uncertainty, identifies inconsistencies in the expert's opinion, takes advantage of the potential benefits of group work while mitigating against the negative effects, and addresses the issues of consensus assessment and consensus building.

Discussion on the papers on 'Elicitation'

Lawrence Phillips (London School of Economics and Political Science) As we sat down to lunch in the staff dining room at University College London about 30 years ago, Dennis Lindley introduced me to a distinguished looking gentleman who to my delight turned out to be Professor Pearson. My youthful enthusiasm for Bayesian statistics, kindled at the University of Michigan in the 1960s by Ward Edwards and Jimmy Savage, led us to discuss the difficulty of obtaining a prior distribution. Professor Pearson gently explained that he could see no way of determining meaningful prior probabilities from the scientists with whom he had worked. The three papers presented today all say, 'Yes, Professor Pearson, it can be done'. However, the authors are providing perspectives that are very different from those of 30 years ago. First, the papers go beyond exploiting the properties of vague prior opinion to justify using non-informative priors, as in the principle of stable estimation (Edwards et al., 1963). For these examples, prior opinion is not vague and it matters. Second, the key question is not how to assess prior opinion, as O'Hagan recognizes; the more general issue is how to obtain meaningful and useful representations of expert uncertainty expressed in probabilistic form. That could apply to priors, likelihoods, posteriors, predictions or indeed any uncertainties. Third, as human beings, experts naturally experience uncertainty as a feeling, not as a numerical probability, so a good elicitation process helps the experts to learn what numbers are appropriate representations of their feelings and minimizes the biases summarized by Kadane and Wolfson. Fourth, we have access to computing power that can reduce the complexities of elicitation and make consistency checking an easy task, as is demonstrated to excellent effect by the University of Durham authors, as well as the others. Finally, all the authors make it clear that the process of assessing expert uncertainty deserves careful attention, in my view at least as much care as is devoted to the design of experiments. Most of us can distinguish a well-designed experiment from a poor one. But would we agree on criteria for judging the acceptability of an elicitation process? These papers suggest several criteria, which I shall summarize, supplemented by my own experience. The most important consideration is that proper elicitation is a sociotechnical process, as demonstrated in the early handbook by Stael von Holstein and Matheson (1979). The interaction between assessor and expert(s) requires careful handling of both social and technical issues at each of the following three main stages in the assessment process.

Eliciting expert beliefs in substantial practical applications [Read before The Royal Statistical Society at ameeting on 'Elicitation' on Wednesday, april 16th, 1997, the President, Professor A. F. M. Smithin the Chair

The practical elicitation of expert beliefs is considered through two contrasting examples. The first example concerns elicitation of engineers' prior beliefs about various quantities relating to the future capital investment need of a water company. Prior beliefs needed to be elicited about very many quantities, but only in the form of prior means, variances and covariances. A computerized procedure was required that could be routinely used by engineers, unsupervised. The second example is a single, application-specific elicitation of the beliefs of hydrogeologists about properties of certain rocks. A full probability specification was ideally to be obtained from a two-day intensive, supervised, elicitation with several experts together. The two different approaches used for these problems are described and contrasted, but a common principle of trying to identify and elicit separately the various sources of expert uncertainty is identified

Analyzing Uncertainties in Experts' Opinions of Forest Plan Performance

Abstract Multi-objective forestry requires new decision support systems to aid the forest owner and foresters in the planning of future treatment schedules. The analytic hierarchy process (AHP), based on pairwise comparison data and Saaty's eigenvector method, is one technique that has been proposed to make such qualitatively different objectives as income from timber sales and scenic beauty of forest landscape commensurable. A weak point of the methodology has been the lack of a statistical theory behind it. We have earlier shown how classical regression techniques can be used to provide a statistical assessment of the uncertainty of the estimated ratio-scales. In this paper we extend the results to a multi-level decision hierarchy commonly used in forest planning. We also provide a Bayesian extension of the regression technique. The advantage of the Bayesian approach is that it provides summaries of expert views that are easily understood by decision makers who may not have extensive understanding of statistical concepts. On the basis of the Bayesian analysis, one can calculate, for example, how likely it is that (in the view of the expert) a given forest plan is better than any other plan being compared. For. Sci. 43(4):521-528.

Public opinion about issues characterized by technological complexity and scientific uncertainty

The ability of Americans to make reasonable, logically consistent assessments about complex policy issues is a longstanding concern among elites and scholars, especially in view of the two most recent elections. Of particular concern, in view of American's low level of scientific literacy, is the public's ability to make such assessments about issues characterized by technological complexity and areas where experts cannot, with confidence, assess the degree of risk of certain phenomena or policy options. Public opinion about genetics research, the use of alar, and the safety of nuclear power plants are three recent examples. The article reports on an experimental study designed to explore whether the public can make such assessments and to identify the factors that enhance the process. The threat of global warming and the safe disposal of solid waste were used as proxies for an array of issues that are technologically complex and characterized by expert uncertainty. I argue that the study—in which over 400 people chosen to reflect a cross-section of the population filled out questionnaires before and after learning more about both issues, and which compares those responses to the views of 400 scientists—shows that the public can make a logically consistent assessment about such issues after a minimal educational intervention and a short period of time. The public's judgment about both issues, as measured by the questionnaire used after the intervention, is strikingly similar to the scientists' views. Further, the few areas of divergence seem rooted more in value differences than in expertise. The article discusses some conditions that enhance or inhibit the public's ability to make such assessments, and comments on the experiment's relevance to practical political decision-making.

A MEASURE OF AVERAGE SENSITIVITY FOR FUZZY LOGICS

Experts usually express their uncertainty by words of natural languages (like "perhaps", "for sure", etc). In the majority of expert systems and intelligent control systems, uncertainty of experts' statements is represented by a number from the interval [0, 1]. There are many different procedures that translate the words that experts use into numbers from [0, 1]. For one and the same word, different procedures can lead to different numbers. Some &– and ∨–operations are very sensitive to this difference in the sense that small changes in t(A) and t(B) can lead to absolutely different estimates for t(A&B) and t(A ∨ B). In view of that, it is reasonable to restrict ourselves to the operations that are the least sensitive to such changes. In this paper, we prove that ab and a + b − ab are "in the average" the least sensitive &– and ∨–operations. This result is in good accordance with the experimental data according to which in. many cases, these operations provide the best description of how experts actually think. We also show how this idea can be applied to other logical connectives (e.g., "not"), and to the choice of membership functions.

Scientists' Uncertainty and Disagreement about Global Climate Change: A Psychological Perspective

Since there are major gaps in scientific knowledge about the greenhouse effect, policy makers are forced to rely on scientists' judgments about the rate of climate change and its effects. It now appears that significant policy decisions will have to be made despite continuing uncertainty and disagreement among scientists. The best that policy makers can do is look for guidance in the best available expert judgment. But when the best experts express great uncertainty and disagree, as seems to be the case with the greenhouse effect, policy makers are unsure about how to proceed. Despite its importance, scientific judgment about the greenhouse effect has not been systematically studied. Psychological research on expert judgment and decision making provides theory, methods, and empirical results that can be used to study judgments based on uncertain and incomplete data and can help to explain the sources of expert uncertainty and disagreement regarding the greenhouse effect. Possible explanations are discussed.

Expert panels and medical uncertainty

Expert panels and medical uncertainty