Sensitivity Analysis Methodology Research Articles

Use of sensitivity and uncertainty analysis in reactor design verification. Part I: TSUNAMI analysis

As for any new reactor design, the ACR-1000® design has to go through a comprehensive design verification process. One of the activities for supporting the ACR physics design calculations using the ACR physics code toolset, namely WIMS-AECL/DRAGON/RFSP, is to compare the flux distributions resulting from the calculation using this toolset at various power calibration monitor (PCM) detector locations against the flux measurement data from the Japanese Advanced Thermal Reactor (ATR) FUGEN. The discussion of this particular design verification exercise will be presented in a two-part paper. The usage of data from the FUGEN reactor qualifies this exercise as design verification by alternate analysis. In order to have meaningful results at the end of the design verification process, the similarity between the ACR-1000 and FUGEN reactors has to be demonstrated. It is accomplished through the sensitivity and uncertainty analysis using the TSUNAMI (Tools for Sensitivity and Uncertainty Analysis Methodology Implementation) methodology. This is the topic of the first part of the paper. In the second part, the results from flux distribution comparison will be presented. Favourable results from this design verification exercise give a high level of confidence that using the same physics toolset in calculating the flux distribution for ACR-1000 reactor will produce results with acceptable fidelity.

Use of sensitivity and uncertainty analysis in reactor design verification Part II: Flux measurement analysis

As for any new reactor design, the ACR-1000® design has to go through a comprehensive design verification process. One of the activities for supporting the ACR physics design calculations using the ACR physics code toolset, namely WIMS-AECL/DRAGON/RFSP, is to compare the flux distributions resulting from the calculation using this toolset at various power calibration monitor (PCM) detector locations against the flux measurement data from the Japanese Advanced Thermal Reactor (ATR) FUGEN. The discussion of this particular design verification exercise will be presented in a two-part paper. The usage of data from the FUGEN reactor qualifies this exercise as design verification by alternate analysis. In order to have meaningful results at the end of the design verification process, the similarity between the ACR-1000 and FUGEN reactors has to be demonstrated. It is accomplished through the sensitivity and uncertainty analysis using the TSUNAMI (Tools for Sensitivity and Uncertainty Analysis Methodology Implementation) methodology. The results from the similarity comparison have been presented in Part I of the paper. In Part II, results from flux distribution comparison will be presented. Favourable results from this design verification exercise give a high level of confidence that using the same physics toolset in calculating the flux distribution for ACR-1000 reactor will produce results with acceptable fidelity. In addition, the results will also give an indication of expected margins in the design calculations, not only at the locations of the PCM detectors but also at the derived bundle and channel powers obtained through the flux mapping calculation.

Direct differentiation method for response sensitivity analysis of a bounding surface plasticity soil model

Finite element (FE) response sensitivity analysis is an important component in gradient-based structural optimization, reliability analysis, system identification, and FE model updating. In this paper, the FE response sensitivity analysis methodology based on the direct differentiation method (DDM) is applied to a bounding surface plasticity material model that has been widely used to simulate nonlinear soil behavior under static and dynamic loading conditions. The DDM-based algorithm is derived and implemented in the general-purpose nonlinear finite element analysis program OpenSees. The algorithm is validated through simulation of the nonlinear cyclic response of a soil element and a liquefiable soil site at Port Island, Japan, under earthquake loading. The response sensitivity results are compared and validated with those obtained from Forward Finite Difference (FFD) analysis. Furthermore, the results are used to determine the relative importance of various soil constitutive parameters to the dynamic response of the system. The DDM-based algorithm is demonstrated to be accurate and efficient in computing the FE response sensitivities, and has great potential in the sensitivity analysis of nonlinear dynamic soil-structure systems.

Methodology for global sensitivity analysis of consequence models

A methodology is presented for global sensitivity analysis of consequence models used in process safety applications. It involves running a consequence model around a hundred times and using the results to construct a statistical emulator, which is essentially a sophisticated curve fit to the data. The emulator is then used to undertake the sensitivity analysis and identify which input parameters (e.g. operating temperature and pressure, wind speed) have a significant effect on the chosen output (e.g. vapour cloud size). Performing the sensitivity analysis using the emulator rather than the consequence model itself leads to significant savings in computing time.To demonstrate the methodology, a global sensitivity analysis is performed on the Phast consequence model for discharge and dispersion. The scenarios studied consist of above-ground, horizontal, steady-state discharges of dense-phase carbon dioxide (CO2), with orifices ranging in diameter from ½ to 2 inch and the liquid CO2 stagnation conditions maintained at between 100 and 150 bar. These scenarios are relevant in scale to leaks from large diameter above-ground pipes or vessels.Seven model input parameters are varied: the vessel temperature and pressure, orifice size, wind speed, humidity, ground surface roughness and height of the release. The input parameters that have a dominant effect on the dispersion distance of the CO2 cloud are identified, both in terms of their direct effect on the dispersion distance and their indirect effect, through interactions with other varying input parameters.The analysis, including the Phast simulations, runs on a standard office laptop computer in less than 30 min. Tests are performed to confirm that a hundred Phast runs are sufficient to produce an emulator with an acceptable degree of accuracy. Increasing the number of Phast runs is shown to have no effect on the conclusions of the sensitivity analysis.The study demonstrates that Bayesian analysis of model sensitivity can be conducted rapidly and easily on consequence models such as Phast. There is the potential for this to become a routine part of consequence modelling.

A relative entropy rate method for path space sensitivity analysis of stationary complex stochastic dynamics

We propose a new sensitivity analysis methodology for complex stochastic dynamics based on the relative entropy rate. The method becomes computationally feasible at the stationary regime of the process and involves the calculation of suitable observables in path space for the relative entropy rate and the corresponding Fisher information matrix. The stationary regime is crucial for stochastic dynamics and here allows us to address the sensitivity analysis of complex systems, including examples of processes with complex landscapes that exhibit metastability, non-reversible systems from a statistical mechanics perspective, and high-dimensional, spatially distributed models. All these systems exhibit, typically non-Gaussian stationary probability distributions, while in the case of high-dimensionality, histograms are impossible to construct directly. Our proposed methods bypass these challenges relying on the direct Monte Carlo simulation of rigorously derived observables for the relative entropy rate and Fisher information in path space rather than on the stationary probability distribution itself. We demonstrate the capabilities of the proposed methodology by focusing here on two classes of problems: (a) Langevin particle systems with either reversible (gradient) or non-reversible (non-gradient) forcing, highlighting the ability of the method to carry out sensitivity analysis in non-equilibrium systems; and, (b) spatially extended kinetic Monte Carlo models, showing that the method can handle high-dimensional problems.

SENSITIVITY ANALYSIS FOR STUDYING THE RELATION BETWEEN BIOCHEMICAL REACTIONS AND METABOLIC PHENOTYPES

Metabolic models are the most widespread type of models in systems biology and are currently used in several applications, including metabolic engineering and investigations of pathological states in which metabolic disorders play a relevant role. Once a model has been defined and corroborated, sensitivity analysis techniques can be used to study the model behavior in relation to perturbations of the model parameters. Here, we describe how it is possible to combine regionalized sensitivity analysis and response surface methodology to screen and quantitatively characterize the relation between metabolic phenotypes and biochemical reactions rates. By means of this approach, we identified the most important reactions for the citric acid efflux from mitochondria, one of the key metabolic traits of cancer cells.

Global Sensitivity Analysis of Jointed Plain Concrete Pavement Mechanistic–Empirical Performance Predictions

The new AASHTO Mechanistic-Empirical Pavement Design Guide (MEPDG) provides pavement analysis and performance predictions for various what-if scenarios. MEPDG performance predictions for anticipated climatic and traffic conditions will depend on the values of the input parameters that characterize pavement materials, layers, design features, and condition. A comprehensive global sensitivity analysis methodology is proposed for evaluating performance predictions for jointed plain concrete pavement to MEPDG inputs for five climatic conditions and three traffic levels. MEPDG inputs evaluated in the analysis include traffic volume, layer thicknesses, material properties, groundwater depth, and geometric parameters. Correlations between MEPDG inputs were considered as appropriate. The global sensitivity analysis varied all inputs simultaneously across the problem domain for each of the 15 base cases (five climates x three traffic levels). Two response surface modeling approaches, multivariate linear regressions and artificial neural networks, were developed for evaluation of MEPDG input sensitivities across the problem domain. The response surface modeling approaches based on artificial neural networks not only provided robust and accurate representations of the complex relationships between MEPDG inputs and distress outputs but also captured the variation in sensitivity across the problem domain. The normalized sensitivity index for the design limit proposed in the study provides practical interpretation of sensitivity by relating a given percentage change in an MEPDG design input to the corresponding percentage change in predicted distress relative to its design limit value.

Hierarchical multivariate regression-based sensitivity analysis reveals complex parameter interaction patterns in dynamic models

Dynamic models of biological systems often possess complex and multivariate mappings between input parameters and output state variables, posing challenges for comprehensive sensitivity analysis across the biologically relevant parameter space. In particular, more efficient and robust ways to obtain a solid understanding of how the sensitivity to each parameter depends on the values of the other parameters are sorely needed.We report a new methodology for global sensitivity analysis based on Hierarchical Cluster-based Partial Least Squares Regression (HC-PLSR)-based approximations (metamodelling) of the input–output mappings of dynamic models, which we expect to be generic, efficient and robust, even for systems with highly nonlinear input–output relationships. The two-step HC-PLSR metamodelling automatically separates the observations (here corresponding to different combinations of input parameter values) into groups based on the dynamic model behaviour, then analyses each group separately with Partial Least Squares Regression (PLSR). This produces one global regression model comprising all observations, as well as regional regression models within each group, where the regression coefficients can be used as sensitivity measures. Thereby a more accurate description of complex interactions between inputs to the dynamic model can be revealed through analysis of how a certain level of one input parameter affects the model sensitivity to other inputs. We illustrate the usefulness of the HC-PLSR approach on a dynamic model of a mouse heart muscle cell, and demonstrate how it reveals interaction patterns of probable biological significance not easily identifiable by a global regression-based sensitivity analysis alone.Applied for sensitivity analysis of a complex, high-dimensional dynamic model of the mouse heart muscle cell, several interactions between input parameters were identified by the two-step HC-PLSR analysis that could not be detected in the single-step global analysis. Hence, our approach has the potential to reveal new biological insight through the identification of complex parameter interaction patterns. The HC-PLSR metamodel complexity can be adjusted according to the nonlinear complexity of the input–output mapping of the analysed dynamic model through adjustment of the number of regional regression models included. This facilitates sensitivity analysis of dynamic models of varying complexities.

Parameter sensitivity analysis of stochastic models: Application to catalytic reaction networks

A general numerical methodology for parametric sensitivity analysis is proposed, which allows to determine the parameters exerting the greatest influence on the output of a stochastic computational model, especially when the knowledge about the actual value of a parameter is insufficient. An application of the procedure is performed on a model of protocell, in order to detect the kinetic rates mainly affecting the capability of a catalytic reaction network enclosed in a semi-permeable membrane to retain material from its environment and to generate a variety of molecular species within its boundaries. It is shown that the former capability is scarcely sensitive to variations in the model parameters, whereas a kinetic rate responsible for profound modifications of the latter can be identified and it depends on the specific reaction network. A faster uptaking of limited resources from the environment may have represented a significant advantage from an evolutionary point of view and this result is a first indication in order to decipher which kind of structures are more suitable to achieve a viable evolution.

Weighted Average Correlation Matrices Method for Correlation Stress Testing and Sensitivity Analysis

Until the onset of recent financial crises, the relative utility of correlation stress testing and sensitivity analysis were commonly underestimated. Changes in correlation can seriously influence positions of both market risk and credit risk. This article aims to present the use of a weighted average correlation matrix as a new and simple methodology for correlation stress testing and sensitivity analysis. This method allows a sensitivity level to be set to a given percentage for both upward and downward adjustments of a correlation matrix, which can be applied to the analysis of both market and credit portfolios. To the best of my knowledge, for the first time a systematic methodology that allows for the downward adjustment of an overall correlation matrix has been proposed. Its simplicity and utility give this methodology great potential to become a common practice for correlation stress testing and sensitivity analysis in the future.

Inactivating ion channels augment robustness of subthreshold intrinsic response dynamics to parametric variability in hippocampal model neurons

Voltage-gated ion channels play a critical role in regulating neuronal intrinsic response dynamics (IRD). Here, we computationally analysed the roles of the two inactivating subthreshold conductances (A and T), individually and in various combinations with the non-inactivating h conductance, in regulating several physiological IRD measurements in the theta frequency range. We found that the independent presence of a T conductance, unlike that of an h conductance, was unable to sustain an inductive phase lead in the theta frequency range, despite its ability to mediate theta frequency resonance. The A conductance, on the other hand, when expressed independently, acted in a manner similar to a leak conductance with reference to most IRD measurements. Next, analysing the impact of pair-wise coexpression of these channels, we found that the coexpression of the h and T conductances augmented the range of parameters over which they sustained resonance and inductive phase lead. Additionally, coexpression of the A conductance with the h or the T conductance elicited changes in IRD measurements that were similar to those obtained with the expression of a leak conductance with a resonating conductance. Finally, to understand the global sensitivity of IRD measurements to all parameters associated with models expressing all three channels, we generated 100,000 neuronal models, each built with a unique set of parametric values. We categorized valid models among these by matching their IRD measurements with experimental counterparts, and found that functionally similar models could be achieved even when underlying parameters displayed tremendous variability and exhibited weak pair-wise correlations. Our results suggest that the three prominent subthreshold conductances contribute differently to intrinsic excitability and to phase coding. We postulate that the differential expression and activity-dependent plasticity of these conductances contribute to robustness of subthreshold IRD, whereby response homeostasis is achieved by recruiting several non-unique combinations of these channel parameters.

A new methodology for sensitivity and stability analysis of analytic network models

In this paper we develop a methodology to study the sensitivity and the stability of models built using the Analytic Network Process. We study two types of stability: core and solution stability. The former deals with finding the region of the perturbation space in which the initial solution (i.e., the alternative that is ranked first) obtained from the ANP model remains most preferred. The latter deals with finding the regions of the perturbation space in which the solutions that were not initially most preferred (i.e., alternatives that were not ranked first) become most preferred (i.e., they are ranked first). The methodology consists of three stages: generation of the perturbation space, finding the boundaries of the regions in the perturbation space in which the different alternatives are ranked first, and finding the stability regions.

Partially identified prevalence estimation under misclassification using the kappa coefficient

We discuss prevalence estimation under misclassification. That is we are concerned with the estimation of a proportion of units having a certain property (being diseased, showing deviant behavior, etc.) from a random sample when the true variable of interest cannot be observed, but a related proxy variable (e.g. the outcome of a diagnostic test) is available. If the misclassification probabilities were known then unbiased prevalence estimation would be possible. We focus on the frequent case where the misclassification probabilities are unknown but two independent replicate measurements have been taken. While in the traditional precise probabilistic framework a correction from this information is not possible due to non-identifiability, the imprecise probability methodology of partial identification and systematic sensitivity analysis allows to obtain valuable insights into possible bias due to misclassification. We derive tight identification intervals and corresponding confidence regions for the true prevalence, based on the often reported kappa coefficient, which condenses the information of the replicates by measuring agreement between the two measurements. Our method is illustrated in several theoretical scenarios and in an example from oral health on prevalence of caries in children.

A methodology for global-sensitivity analysis of time-dependent outputs in systems biology modelling

One of the main challenges in the development of mathematical and computational models of biological systems is the precise estimation of parameter values. Understanding the effects of uncertainties in parameter values on model behaviour is crucial to the successful use of these models. Global sensitivity analysis (SA) can be used to quantify the variability in model predictions resulting from the uncertainty in multiple parameters and to shed light on the biological mechanisms driving system behaviour. We present a new methodology for global SA in systems biology which is computationally efficient and can be used to identify the key parameters and their interactions which drive the dynamic behaviour of a complex biological model. The approach combines functional principal component analysis with established global SA techniques. The methodology is applied to a model of the insulin signalling pathway, defects of which are a major cause of type 2 diabetes and a number of key features of the system are identified.

Novel global sensitivity analysis methodology accounting for the crucial role of the distribution of input parameters: application to systems biology models

SUMMARYThe reliability of model predictions is affected by multiple sources of uncertainty; therefore, most of the efforts for modeling biological systems include a sensitivity analysis step aiming to identify the key contributors to uncertainty. This generates insight about the robustness of the model to variations in environmental conditions, kinetic parameters, initial concentration of the species, or any other source of uncertainty. Local sensitivities measure the robustness of the model to small perturbations on the inputs around their nominal value. There are several numerical methods for the calculation of local sensitivities, but the calculated values should be identical within the numerical accuracy of the method used. In contrast, as will be shown in this contribution, the results of different global sensitivity analysis methods can be very different and highly dependent on the distribution considered for the inputs under evaluation. In this work, derivative‐based global sensitivities are extended to be able to consider an accurate probability density function for the parameters based on the likelihood function. This strategy enforces the areas of the parameter space most likely to reproduce the desired behavior, minimizing the importance of parameter sets with low probability of being optimal to dominate the sensitivity ranking. A model of a biochemical pathway with three enzymatic steps is used here to illustrate the performance of several relevant global sensitivity analysis methods considering different probability density functions for the parameters and revealing important hints about which method and distribution to choose for each type of model and purpose of the analysis. Copyright © 2012 John Wiley & Sons, Ltd.

Sensitivity analysis for multi‐attribute system selection problems in onshore Environmentally Friendly Drilling (EFD)

AbstractInput data used in Multi‐Attribute Decision Making (MADM) problems are often perceived to be imprecise by decision‐makers because they are based on expert assessments. As a result, an important step in many applications of MADM is to perform a sensitivity analysis on the input data to help decision‐makers understand how in which regions of the input data space they can be most confident in the recommended decisions and where improved input information is most needed. This paper describes sensitivity analysis procedures which will help decision‐makers examine the robustness of the optimal solution to changes in input parameters in system selection decisions, where the system is made up of components (drilling technologies) that can be combined in many different ways. This paper presents two different sensitivity analysis methodologies. One is a sensitivity analysis for weighting factors of each attribute, and the other is a sensitivity analysis for uncertainty of overall attribute inputs. An application case study of the proposed approach is described to select optimal onshore Environmentally Friendly Drilling (EFD) systems at Green Lake near McFaddin, TX. © 2011 Wiley Periodicals, Inc.

Global sensitivity analysis of stochastic computer models with joint metamodels

The global sensitivity analysis method used to quantify the influence of uncertain input variables on the variability in numerical model responses has already been applied to deterministic computer codes; deterministic means here that the same set of input variables always gives the same output value. This paper proposes a global sensitivity analysis methodology for stochastic computer codes, for which the result of each code run is itself random. The framework of the joint modeling of the mean and dispersion of heteroscedastic data is used. To deal with the complexity of computer experiment outputs, nonparametric joint models are discussed and a new Gaussian process-based joint model is proposed. The relevance of these models is analyzed based upon two case studies. Results show that the joint modeling approach yields accurate sensitivity index estimators even when heteroscedasticity is strong.

Mathematical models for the study of HIV spread and control amongst men who have sex with men

For a quarter of century, mathematical models have been used to study the spread and control of HIV amongst men who have sex with men (MSM). We searched MEDLINE and EMBASE databases up to the end of 2010 and reviewed this literature to summarise the methodologies used, key model developments, and the recommended strategies for HIV control amongst MSM. Of 742 studies identified, 127 studies met the inclusion criteria. Most studies employed deterministic modelling methods (80%). Over time we saw an increase in model complexity regarding antiretroviral therapy (ART), and a corresponding decrease in complexity regarding sexual behaviours. Formal estimation of model parameters was carried out in only a small proportion of the studies (22%) while model validation was considered by an even smaller proportion (17%), somewhat reducing confidence in the findings from the studies. Nonetheless, a number of common conclusions emerged, including (1) identification of the importance of assumptions regarding changes in infectivity and sexual contact rates on the impact of ART on HIV incidence, that subsequently led to empirical studies to gather these data, and (2) recommendation that multiple strategies would be required for effective HIV control amongst MSM. The role of mathematical models in studying epidemics is clear, and the lack of formal inference and validation highlights the need for further developments in this area. Improved methodologies for parameter estimation and systematic sensitivity analysis will help generate predictions that more fully express uncertainty, allowing better informed decision making in public health.

Parameter Sensitivity of the Arctic Biome–BGC Model for Estimating Evapotranspiration in the Arctic Coastal Plain

Previous research has indicated that modeling evapotranspiration (ET) in the Arctic Coastal Plain is challenging due to unique ecosystem conditions which include mosses, permafrost, and standing dead vegetation. A new version of the commonly used Biome–BGC (Biogeochemical Cycles) model (Arctic Biome–BGC) was developed that included: (1) a water storage and vertical drainage/infiltration routine that accounts for permafrost and mosses, (2) a modified representation of energy available at the surface which includes ground heat flux and simulates interception of incoming radiation by standing dead vegetation, and (3) a background evaporation routine that allows for moss and open water evaporation. In this study we investigated the sensitivity of model predictions to variations in parameter values, and to provide a conceptual validation of Arctic Biome–BGC. Using the generalized sensitivity analysis methodology, 13 parameters were evaluated. Results indicate that the model was sensitive to 8 of the 13 parameters. Seven of these parameters were introduced in the development of Arctic Biome–BGC and related to both energy reaching the ground surface and the amount of water stored within the soil and moss layers. The remaining sensitive parameter modulates the rate of snowmelt. These results validate the conceptual modifications included in the Arctic Biome–BGC model for estimating ET.

An effective sensitivity analysis methodology for the reliability‐based structural optimization design to high temperature components

AbstractMost of physical parameters of high temperature components are usually correlated with the temperature parameter, which makes the reliability‐based structural optimization design more complicated. So far, few efficient sensitivity analysis methods adapted to highly nonlinear performance function with correlated non‐normal variables have been invented. An effective simulation approach is presented to perform reliability sensitivity analysis with the aid of Cholesky factorization and curve fitting. Moreover, in order to improve the programming and computational efficiency, the mixed programming technique of Visual Basic and MATLAB was employed to develop a reliability sensitivity analysis program. The numerical case shows that the results are consistent with those from other traditional methods, and the engineering cases indicate that the proposed method can be effectively used in highly nonlinear function with correlated non‐normal parameters. Meanwhile, the results also demonstrate that the correlation coefficient parameter of high temperature components is very important in the reliability‐based structural optimization design; hence, this parameter should be fully taken into consideration to ensure that the analysis results are accurate and reasonable. Copyright © 2011 John Wiley & Sons, Ltd.