Uncertainty Analysis Techniques Research Articles

Scaling and uncertainty: in search of conceptual linkages and practical approaches

The authors of Scaling and Uncertainty Analysis in Ecology have cast a broad net in reviewing and synthesizing these two all-encompassing topics. Scaling has become a major ecological issue in the fifteen years since Levin (1992) defined the problem of pattern and scale as a central theme in ecology, as evidenced by the number of books on this topic that have already been published (e.g., Schnieder 1994; Bissonette 1997; Peterson and Parker 1998; Waring and Running 1998; Gardner et al. 2001). Similarly, the need to quantify uncertainty has been the driving force behind the development and application of statistical methods in ecology, and uncertainty analysis has always been an important aspect of ecological simulation modeling. As with scaling, uncertainty is a pervasive theme in nearly all ecological research and has been the subject of recent books focusing on spatial uncertainty in ecology and natural resource management (Mowrer and Congalton 2000; Hunsaker et al. 2001). Scaling and Uncertainty Analysis in Ecology takes the unique perspective of addressing these two important topics simultaneously, providing an opportunity to consider how they are interlinked. The first section of the book consists of several background chapters describing concepts and methods related to scaling and uncertainty. Chapter 1 (Wu and Li) provides a useful framework for understanding the multitude of scale-related concepts and definitions based on a three-tiered model of dimensions, kinds, and components of scale. Chapter 2 (Wu and Li) similarly characterizes and classifies the different quantitative methods and models used in scaling research. The next two chapters provide background on the topic of uncertainty. Chapter 3 (Li and Wu) primarily addresses techniques for uncertainty analysis of ecological simulation models. Chapter 4 (Berk and de Leeuw) provides an overview of statistical approaches for analyzing hierarchical, multi-scale datasets (e.g. patches nested within landscapes) and spatially autocorrelated phenomena. Additional chapters in the concepts and methods section address downscaling species distribution patterns to estimate local abundance (Chapter 5, He and Reed), upscaling experimental results from model systems to real landscapes (Chapter 6, Bradford and Reynolds), and upscaling results from plot-level studies to heterogeneous landscapes (Chapter 7, Peters et al.). Finally, Chapter 8 (Wessman and Bateson) provide a thorough discussion of scale-related considerations in applying satellite remote sensing to measure earth-surface patterns and processes. The second section of Scaling and Uncertainty Analysis in Ecology provides several case studies that illustrate the myriad ways in which scaling techniques M. C. Wimberly (&) Geographic Information Science Center of Excellence, South Dakota State University, Brookings, SD 57007, USA e-mail: michael.wimberly@sdstate.edu

Incorporating Reliability into the Definition of the Margin of Safety in Total Maximum Daily Load Calculations

In the calculation of the total maximum daily load (TMDL), the margin of safety (MOS) is an attempt to account for the uncertainties related to the conceptual representation of the system and the inherently random nature of the processes under study. In general, the MOS is often the result of arbitrary or poorly supported decisions that have little to do with the reliability of the system. To improve the current practice in TMDL calculations, an enhanced reliability-based MOS is proposed here. The Modified Rosenblueth point estimate method is proposed in this paper as a valuable means to assess the uncertainty of a system, and, thus, its probability of failure. The Modified Rosenblueth method is a less computationally demanding uncertainty analysis technique, and can be easily coupled with deterministic hydraulic and water quality numerical models. A case study on arsenic concentrations in the Niagara River is discussed in this paper. The results of the uncertainty analysis are used to compute the effecti...

MyLake—A multi-year lake simulation model code suitable for uncertainty and sensitivity analysis simulations

Uncertainty and sensitivity analysis provide powerful means to enhance the performance of the models, to increase the transparency and credibility of the model results, and to allow the decision-maker to judge whether the model results are sufficiently accurate and precise to support decision-making or not. Uncertainty and sensitivity analysis of numerical models usually require hundreds, thousands, or even more, repeated model runs (e.g., in Monte Carlo simulation) as well as access to change the model parameters and variables on each model run. Consequently, many model codes, although widely used and well-formulated, are not well-suited for uncertainty and sensitivity analysis due to too long model execution time or due to lack of suitable interface to change model parameters and variables in an automated way. In this paper, we describe in details a lake model code, which aims to combine good simulation capabilities with efficient model execution time and easy application of numerical uncertainty and sensitivity analysis techniques. This model code is called MyLake (Multi-year Lake simulation model) and it is a one-dimensional process-based model code for simulation of daily vertical distribution of lake water temperature and thus density stratification, evolution of seasonal lake ice and snow cover, sediment–water interactions, and phosphorus-phytoplankton dynamics. After giving a detailed technical description of the different processes and algorithms included in the current version 1.2 of the MyLake model code, we present some results from a model application example including an Extended Fourier Amplitude Sensitivity Test (Extended FAST) sensitivity analysis and a simple Monte Carlo simulation based uncertainty analysis. Finally we discuss the performance of the MyLake code, especially in connection with numerical uncertainty and sensitivity analysis techniques.

Uncertainty analysis of statistically downscaled temperature and precipitation regimes in Northern Canada

Uncertainty analysis is used to make a quantitative evaluation of the reliability of statistically downscaled climate data representing local climate conditions in the northern coastlines of Canada. In this region, most global climate models (GCMs) have inherent weaknesses to adequately simulate the climate regime due to difficulty in resolving strong land/sea discontinuities or heterogeneous land cover. The performance of the multiple regression-based statistical downscaling model in reproducing the observed daily minimum/maximum temperature, and precipitation for a reference period (1961–1990) is evaluated using climate predictors derived from NCEP reanalysis data and those simulated by two coupled GCMs (the Canadian CGCM2 and the British HadCM3). The Wilcoxon Signed Rank test and bootstrap confidence-interval estimation techniques are used to perform uncertainty analysis on the downscaled meteorological variables. The results show that the NCEP-driven downscaling results mostly reproduced the mean and variability of the observed climate very well. Temperatures are satisfactorily downscaled from HadCM3 predictors while some of the temperatures downscaled from CGCM2 predictors are statistically significantly different from the observed. The uncertainty in precipitation downscaled with CGCM2 predictors is comparable to the ones downscaled from HadCM3. In general, all downscaling results reveal that the regression-based statistical downscaling method driven by accurate GCM predictors is able to reproduce the climate regime over these highly heterogeneous coastline areas of northern Canada. The study also shows the applicability of uncertainty analysis techniques in evaluating the reliability of the downscaled data for climate scenarios development.

Stochastic Markovian modeling of electrophysiology of ion channels: Reconstruction of standard deviations in macroscopic currents

Markovian models of ion channels have proven useful in the reconstruction of experimental data and prediction of cellular electrophysiology. We present the stochastic Galerkin method as an alternative to Monte Carlo and other stochastic methods for assessing the impact of uncertain rate coefficients on the predictions of Markovian ion channel models. We extend and study two different ion channel models: a simple model with only a single open and a closed state and a detailed model of the cardiac rapidly activating delayed rectifier potassium current. We demonstrate the efficacy of stochastic Galerkin methods for computing solutions to systems with random model parameters. Our studies illustrate the characteristic changes in distributions of state transitions and electrical currents through ion channels due to random rate coefficients. Furthermore, the studies indicate the applicability of the stochastic Galerkin technique for uncertainty and sensitivity analysis of bio-mathematical models.

Integration of stochastic generation in power systems

Stochastic generation, i.e., electrical power production by an uncontrolled primary energy source, is expected to play an important role in future power systems. A new power system structure is created due to the large-scale implementation of this small-scale, distributed, non-dispatchable generation; the ‘ horizontally-operated’ system. Modeling methodologies that can deal with the operational uncertainty introduced by these power units should be used for analyzing the impact of this generation to the system. In this contribution, the principles for this modeling are presented, based on the decoupling of the single stochastic generator behavior (marginal distribution-stochastic unit capacity) from the concurrent behavior of the stochastic generators (stochastic dependence structure-stochastic system dispatch). Subsequently, the stochastic bounds methodology is applied to model the extreme power contribution of the stochastic generation to the system, based on two new sampling concepts ( comonotonicity–countermonotonicity). The application of this methodology to the power system leads to the definition of clusters of positively correlated stochastic generators and the combination of different clusters based on the sampling concepts. The stochastic decomposition and clustering concepts presented in this contribution provide the basis for the application of new uncertainty analysis techniques for the modeling of stochastic generation in power systems.

Estimation procedures and error analysis for inferring the total plutonium (Pu) produced by a graphite-moderated reactor

Graphite isotope ratio method (GIRM) is a technique that uses measurements and computer models to estimate total plutonium (Pu) production in a graphite-moderated reactor. First, isotopic ratios of trace elements in extracted graphite samples from the target reactor are measured. Then, computer models of the reactor relate those ratios to Pu production. Because Pu is controlled under non-proliferation agreements, an estimate of total Pu production is often required, and a declaration of total Pu might need to be verified through GIRM. In some cases, reactor information (such as core dimensions, coolant details, and operating history) are so well documented that computer models can predict total Pu production without the need for measurements. However, in most cases, reactor information is imperfectly known, so a measurement and model-based method such as GIRM is essential. Here, we focus on GIRM's estimation procedure and its associated uncertainty. We illustrate a simulation strategy for a specific reactor that estimates GIRM's uncertainty and determines which inputs contribute most to GIRM's uncertainty, including inputs to the computer models. These models include a “local” code that relates isotopic ratios to the local Pu production, and a “global” code that predicts the Pu production shape over the entire reactor. This predicted shape is included with other 3D basis functions to provide a “hybrid basis set” that is used to fit the local Pu production estimates. The fitted shape can then be integrated over the entire reactor to estimate total Pu production. This GIRM evaluation provides a good example of several techniques of uncertainty analysis and introduces new reasons to fit a function using basis functions in the evaluation of the impact of uncertainty in the true 3D shape.

Uncertainty analysis in a numerical modelling context – an example application on a coal mine roadway design

Natural rock's geomechanical properties are inherently variable so geotechnical engineers are unavoidably faced with decision making under a degree of uncertainty. However, a traditional single property value numerical modelling design does not address this uncertainty other than by applying safety factors to the answer or using worst assumptions for parameters. With complex problems, these approaches are often impractical. To address the uncertainty involved in a roadway design, the authors have combined the popular Monte Carlo uncertainty analysis technique with a powerful non-linear numerical modelling method by means of a controlling Visual Basic program. The numerical model used is a 20 148 element representation of a stress-loaded, bolt-supported, coal mine roadway in a state of plastic yield. The combination of the time-consuming looping of the Monte Carlo technique with the already time-intensive non-linear numerical model is challenging and raises a number of issues the authors explore to assist others with this type of analysis. With a practical run-time limit of a month on a fast personal computer, the Monte Carlo analysis approach needs to be carefully controlled as less than 1000 Monte Carlo parameter combinations are available for the analysis. It should be noted that both geotechnical and geometrical parameters are varied in the analysis. The input parameters whose uncertainty has a significant impact on the numerical model output are first screened out by a one-at-a-time analysis and are then treated as random in a follow-up analysis. All the remaining inputs are assigned nominal mean values. The application of this hybrid technique to an underground coal mine roadway design is introduced in this paper. The form and distribution of the analysis results is shown to provide useful uncertainty information about the potential roof deformation of the roadway with respect to input parameters' uncertainty.

Uncertainty and Sensitivity Analysis Techniques as Tools for the Quality Assessment of Composite Indicators

SummaryComposite indicators are increasingly used for bench-marking countries’ performances. Yet doubts are often raised about the robustness of the resulting countries’ rankings and about the significance of the associated policy message. We propose the use of uncertainty analysis and sensitivity analysis to gain useful insights during the process of building composite indicators, including a contribution to the indicators’ definition of quality and an assessment of the reliability of countries’ rankings. We discuss to what extent the use of uncertainty and sensitivity analysis may increase transparency or make policy inference more defensible by applying the methodology to a known composite indicator: the United Nations's technology achievement index.

Deriving architectural flexibility requirements in safety-critical systems

Safety-critical embedded systems are constrained by safety regulations that require the designers of the system to explain its operation. This includes the operation of any flexibility mechanisms present in the design, and the rationale for their inclusion. The ability to place such flexibility where it is most needed is a crucial factor in reducing the cost and risk of safety-critical system development. In this paper an analysis technique that the designer can apply when faced with potential requirements problems is described and evaluated. The technique derives flexibility requirements from indicators of customer uncertainty in the way the requirement is expressed. This allows the designer to quickly describe the required flexibility in the architecture and proceed with design even when the requirement is expected to change. The evaluation shows a significant improvement in the ability of a design to manage change when it contains flexibility that is targeted using the uncertainty analysis technique, compared with flexibility that is generated through more conventional means.

A comparative assessment of efficient uncertainty analysis techniques for environmental fate and transport models: application to the FACT model

This work presents a comparative assessment of efficient uncertainty modeling techniques, including Stochastic Response Surface Method (SRSM) and High Dimensional Model Representation (HDMR). This assessment considers improvement achieved with respect to conventional techniques of modeling uncertainty (Monte Carlo). Given that traditional methods for characterizing uncertainty are very computationally demanding, when they are applied in conjunction with complex environmental fate and transport models, this study aims to assess how accurately these efficient (and hence viable) techniques for uncertainty propagation can capture complex model output uncertainty. As a part of this effort, the efficacy of HDMR, which has primarily been used in the past as a model reduction tool, is also demonstrated for uncertainty analysis. The application chosen to highlight the accuracy of these new techniques is the steady state analysis of the groundwater flow in the Savannah River Site General Separations Area (GSA) using the subsurface Flow And Contaminant Transport (FACT) code. Uncertain inputs included three-dimensional hydraulic conductivity fields, and a two-dimensional recharge rate field. The output variables under consideration were the simulated stream baseflows and hydraulic head values. Results show that the uncertainty analysis outcomes obtained using SRSM and HDMR are practically indistinguishable from those obtained using the conventional Monte Carlo method, while requiring orders of magnitude fewer model simulations.

Sensitivity and uncertainty analysis coupled with automatic calibration for a distributed watershed model

Distributed watershed models should pass through a careful calibration procedure before they are utilized as a decision making aid in the planning and management of water resources. Although manual approaches are still frequently used for calibration, they are tedious, time consuming, and require experienced personnel. This paper describes an automatic approach for calibrating daily streamflow and daily sediment concentration values estimated by the US Department of Agriculture's distributed watershed simulation model, Soil and Water Assessment Tool (SWAT). The automatic calibration methodology applies a hierarchy of three techniques, namely screening, parameterization, and parameter sensitivity analysis, at the parameter identification stage of model calibration. The global parameter sensitivity analysis is conducted using a stepwise regression analysis on rank-transformed input–output data pairs. Latin hypercube sampling is used to generate input data from the assigned distributions and ranges, and parameter estimation is performed using genetic algorithm. The Generalized Likelihood Uncertainty Estimation methodology is subsequently implemented to investigate uncertainty of model estimates, accounting for errors due to model structure, input data and model parameters. To demonstrate their effectiveness, the parameter identification, parameter estimation, model verification, and uncertainty analysis techniques are applied to a watershed located in southern Illinois.

Locating new production wells using a probabilistic-based groundwater model

Groundwater models provide an efficient means to evaluate the sustainability of new groundwater production wells. An important component of any groundwater modeling analysis is the quantification of the prediction uncertainty. This analysis incorporates uncertainty analysis techniques within the groundwater modeling environment to predict regions within a groundwater basin that have a low probability of excessive drawdown and groundwater contamination. The analysis focuses on the quantification of a basin-wide water-balance, generation of a spatial distribution of lithology, locating a nearby total dissolved solids (TDS) plume, and construction of an accurate groundwater flow model. Thousands of Monte Carlo realizations are simulated to determine the most probable parameter sets based on an objective function that minimizes the error between simulated and observed hydraulic head. Only those simulations with small error are used in a subsequent analysis to investigate 77 potential well locations near the city of Fernley, in northern Nevada. The final result is a map depicting the probability that a production well has either excessive drawdown and/or encroachment of the TDSs plume.

地質環境特性評価の不確実性解析手法に関する検討

地質環境特性評価の不確実性解析手法に関する検討

A case study for assessing uncertainty in local-scale regulatory air quality modeling applications

In this paper, we expand upon established uncertainty analysis techniques to demonstrate a general method for assessing variability and uncertainty in Gaussian air pollutant dispersion modeling systems. To illustrate this method, we estimated variability and uncertainty in predicted hexavalent chromium concentrations generated by welding operations at a shipbuilding and repair facility in California. Using Monte Carlo statistical techniques, we propagated uncertainty across both ISCST3 and AERMOD, and estimated the contribution of variability and uncertainty from four model components: emissions, spatial and temporal allocation of emissions, model parameters, and meteorology. Our results indicated the 95% confidence interval uncertainty at each receptor spanned an order of magnitude. From a practical perspective uncertainty is most important at receptors with highest predicted concentrations. In this case study, emissions were the primary source of uncertainty. However, Gaussian models are also sensitive to location of emission releases, meteorology, and model parameters. Simplified modeling approaches may lead to errors in pollutant concentration estimates, especially in close proximity to emissions sources where predicted concentrations are highest.

Latin hypercube sampling and the propagation of uncertainty in analyses of complex systems

The following techniques for uncertainty and sensitivity analysis are briefly summarized: Monte Carlo analysis, differential analysis, response surface methodology, Fourier amplitude sensitivity test, Sobol' variance decomposition, and fast probability integration. Desirable features of Monte Carlo analysis in conjunction with Latin hypercube sampling are described in discussions of the following topics: (i) properties of random, stratified and Latin hypercube sampling, (ii) comparisons of random and Latin hypercube sampling, (iii) operations involving Latin hypercube sampling (i.e. correlation control, reweighting of samples to incorporate changed distributions, replicated sampling to test reproducibility of results), (iv) uncertainty analysis (i.e. cumulative distribution functions, complementary cumulative distribution functions, box plots), (v) sensitivity analysis (i.e. scatterplots, regression analysis, correlation analysis, rank transformations, searches for nonrandom patterns), and (vi) analyses involving stochastic (i.e. aleatory) and subjective (i.e. epistemic) uncertainty.

Integrated assessment and modelling: features, principles and examples for catchment management

To meet the challenges of sustainability and catchment management requires an approach that assesses resource usage options and environmental impacts integratively. Assessment must be able to integrate several dimensions: the consideration of multiple issues and stakeholders, the key disciplines within and between the human and natural sciences, multiple scales of system behaviour, cascading effects both spatially and temporally, models of the different system components, and multiple databases. Integrated assessment (IA) is an emerging discipline and process that attempts to address the demands of decision makers for management that has ecological, social and economic values and considerations. This paper summarises the features of IA and argues the role for models and information systems as a prime activity. Given the complex nature of IA problems, the broad objectives of IA modelling should be to understand the directions and magnitudes of change in relation to management interventions so as to be able to differentiate between associated outcome sets. Central to this broad objective is the need for improved techniques of uncertainty and sensitivity analysis that can provide a measure of confidence in the ability to differentiate between different decisions. Three examples of problems treated with an IA approach are presented. The variations in the way that the different dimensions are integrated in the modelling are discussed to highlight the sorts of choices that can be made in model construction. The conclusions stress the importance of IA as a process, not just as a set of outcomes, and define some of the deficiencies to be overcome.

Efficient Uncertainty Analysis Methods for Multidisciplinary Robust Design

Efficient Uncertainty Analysis Methods for Multidisciplinary Robust Design

Analytical and Experimental Assessment of Two Methods of Determining Turbine Efficiency

Theefe ciencyofaturbinemaybecalculatedfrommeasuredtestvariablesbyseveraldifferentmethods.However, the literature does not contain documentation to aid one in determining the preferred efe ciency method for particular test situations. Two methods commonly used for cold aire ow testing of turbines, the thermodynamic and the mechanical methods, were evaluated both experimentally and using uncertainty analysis techniques. The experimental work involved performance testing of a next-generation, compact, high-turning-angle turbine design in which the e ow contained signie cant high-gradient regions. The turbine efe ciency was calculated from the measured test data using both efe ciency methods, and enhanced uncertainty analysis tools were developed to evaluate the efe ciency uncertainty for each method. The efe ciency uncertainty equations were written in a new form to account properly for conceptual bias and correlated bias terms. The results showed that using the form of the equations developed was essential to obtaining the correct efe ciency uncertainty estimates. The efe ciency uncertainty estimates obtained from the analysis then allowed a comparison of the two methods from which conclusions were drawn to guide in determining the preferred efe ciency method for individual applications.

Integrative risk and uncertainty analysis for complex public sector operational systems

Complex public sector operational systems are affected not only by the equipment and software that are used, but also by the personnel who operate the equipment, the procedures specified, the organizational culture, political considerations, and previous management decisions affecting these factors. Because of system complexity, efforts to conduct risk analyses in such environments have typically focused only on specific parts of the operation and have thus not provided a holistic view of the sources of uncertainty and the impacts of various risk elements. In this paper, an integrative framework for performing risk and uncertainty analysis for complex public sector operational systems is developed and described. A set of nested influence diagrams is used to summarize and characterize the important relationships in the system and the uncertainties affecting system output. The relationships within each influence diagram are then modeled analytically or empirically and appropriate risk and uncertainty analysis techniques are chosen for each system component. The results from the individual analyses are then aggregated to obtain a holistic view of the overall risks. The integrative approach is illustrated through a real case study performed for the US Coast Guard's International Ice Patrol. The case study illustrates how sources of uncertainty are identified, models of subsystems are developed that incorporate the uncertain factors, and analyses are performed and synthesized to assess the overall risks associated with determining the “Limits of All Known Ice.” The case study demonstrates that risk analyses can be successfully performed even for very complex operational systems using the integrative approach.