Global Sensitivity Analysis Techniques Research Articles

Global sensitivity analysis approach for input selection and system identification purposes--a new framework for feedforward neural networks.

A new algorithm for the selection of input variables of neural network is proposed. This new method, applied after the training stage, ranks the inputs according to their importance in the variance of the model output. The use of a global sensitivity analysis technique, extended Fourier amplitude sensitivity test, gives the total sensitivity index for each variable, which allows for the ranking and the removal of the less relevant inputs. Applied to some benchmarking problems in the field of features selection, the proposed approach shows good agreement in keeping the relevant variables. This new method is a useful tool for removing superfluous inputs and for system identification.

Global sensitivity analysis of England's housing energy model

Housing energy models support informed decision-making for energy efficiency and CO2 reduction strategies. However, they are subject to multiple sources of uncertainty. Previous studies have analysed uncertainties using a local one-at-a-time approach, but this suffers from significant limitations. Here two global sensitivity analysis techniques, elementary effects and a variance-based method, have been employed to overcome these limitations. Correct understanding of model sensitivities matters, for interpreting findings and directing research to reduce uncertainties. Analysis of the Cambridge Housing Model, a Standard Assessment Procedure (SAP)-based housing energy model for England, finds good agreement between the global analyses but there are major differences compared with the local analysis. We identify SAP wall U-values and demand temperature as by far the most significant uncertain parameters; along with SAP roof, window and floor U-values these account for 96% of the observed variation in outputs. This could have major implications for national-level estimates of savings based on such models.

Simulation of momentum, heat and mass transfer in direct contact membrane distillation: A computational fluid dynamics approach

Membrane distillation is one of the newest methods for water purification. In this study, the behavior of direct contact membrane distillation (DCMD) process under various operating conditions was mathematically modeled using computational fluid dynamics (CFD) method. The effects of different parameters on transmembrane flux and temperature polarization coefficient (TPC) were measured by global sensitivity analysis technique. The sensitivity analysis results indicated that the most affecting design parameters in DCMD are membrane thickness and feed temperature.

Moment independent sensitivity analysis with correlations

The moment independent importance measure is a popular global sensitivity analysis technique, and aims at evaluating contributions of the inputs to the whole output distribution. In this work, moment independent sensitivity analysis is performed for models with correlated inputs, by decomposing the importance measure into the uncorrelated part and correlated part. The correlated input variables are first orthogonalized, then the moment independent sensitivity analysis of the newly generated independent variables is performed. Discussions indicate that the moment independent importance measures, so obtained, can be interpreted as the full, correlated and uncorrelated contributions of the original inputs to the whole output distribution. Procedure of the proposed approach has been generalized. By the decomposition, the information provided by the moment independent importance analysis is enriched, which has been demonstrated by the application to several examples.

Global sensitivity analysis applied to drying models for one or a population of granules

The development of mechanistic models for pharmaceutical processes is of increasing importance due to a noticeable shift toward continuous production in the industry. Sensitivity analysis is a powerful tool during the model building process. A global sensitivity analysis (GSA), exploring sensitivity in a broad parameter space, is performed to detect the most sensitive factors in two models, that is, one for drying of a single granule and one for the drying of a population of granules [using population balance model (PBM)], which was extended by including the gas velocity as extra input compared to our earlier work. β2 was found to be the most important factor for the single particle model which is useful information when performing model calibration. For the PBM‐model, the granule radius and gas temperature were found to be most sensitive. The former indicates that granulator performance impacts drying behavior, the latter is informative with respect to the variables that primarily need to be controlled during continuous operation. In addition, several GSA techniques are analyzed and compared with respect to the correct conclusion and computational load. © 2014 American Institute of Chemical Engineers AIChE J, 60: 1700–1717, 2014

What Can We Learn from Global Sensitivity Analysis of Biochemical Systems?

Most biological models of intermediate size, and probably all large models, need to cope with the fact that many of their parameter values are unknown. In addition, it may not be possible to identify these values unambiguously on the basis of experimental data. This poses the question how reliable predictions made using such models are. Sensitivity analysis is commonly used to measure the impact of each model parameter on its variables. However, the results of such analyses can be dependent on an exact set of parameter values due to nonlinearity. To mitigate this problem, global sensitivity analysis techniques are used to calculate parameter sensitivities in a wider parameter space. We applied global sensitivity analysis to a selection of five signalling and metabolic models, several of which incorporate experimentally well-determined parameters. Assuming these models represent physiological reality, we explored how the results could change under increasing amounts of parameter uncertainty. Our results show that parameter sensitivities calculated with the physiological parameter values are not necessarily the most frequently observed under random sampling, even in a small interval around the physiological values. Often multimodal distributions were observed. Unsurprisingly, the range of possible sensitivity coefficient values increased with the level of parameter uncertainty, though the amount of parameter uncertainty at which the pattern of control was able to change differed among the models analysed. We suggest that this level of uncertainty can be used as a global measure of model robustness. Finally a comparison of different global sensitivity analysis techniques shows that, if high-throughput computing resources are available, then random sampling may actually be the most suitable technique.

Climate-based archetypes for the environmental fate assessment of chemicals

Emissions of chemicals have been on the rise for years, and their impacts are greatly influenced by spatial differentiation. Chemicals are usually emitted locally but their impact can be felt both locally and globally, due to their chemical properties and persistence. The variability of environmental parameters in the emission compartment may affect the chemicals' fate and the exposure at different orders of magnitude. The assessment of the environmental fate of chemicals and the inherent spatial differentiation requires the use of multimedia models at various levels of complexity (from a simple box model to complex computational and high-spatial-resolution models). The objective of these models is to support ecological and human health risk assessment, by reducing the uncertainty of chemical impact assessments. The parameterisation of spatially resolved multimedia models is usually based on scenarios of evaluative environments, or on geographical resolutions related to administrative boundaries (e.g. countries/continents) or landscape areas (e.g. watersheds, eco-regions). The choice of the most appropriate scale and scenario is important from a management perspective, as a balance should be reached between a simplified approach and computationally intensive multimedia models. In this paper, which aims to go beyond the more traditional approach based on scale/resolution (cell, country, and basin), we propose and assess climate-based archetypes for the impact assessment of chemicals released in air. We define the archetypes based on the main drivers of spatial variability, which we systematically identify by adopting global sensitivity analysis techniques. A case study that uses the high resolution multimedia model MAPPE (Multimedia Assessment of Pollutant Pathways in the Environment) is presented. Results of the analysis showed that suitable archetypes should be both climate- and chemical-specific, as different chemicals (or groups of them) have different traits that influence their spatial variability. This hypothesis was tested by comparing the variability of the output of MAPPE for four different climatic zones on four different continents for four different chemicals (which represent different combinations of physical and chemical properties). Results showed the high suitability of climate-based archetypes in assessing the impacts of chemicals released in air. However, further research work is still necessary to test these findings.

An investigation into future performance and overheating risks in Passivhaus dwellings

In response to UK government policy mandating the construction of ‘zero carbon’ homes by 2016 there have been significant changes in the way dwellings are being designed and built. Recent years have seen a rapid uptake in the adoption of the German Passivhaus standard as a template for ultra-low energy and zero carbon buildings in the UK. Despite genuine motivations to mitigate climate change and fuel poverty there is a lack of research investigating the long-term performance of Passivhaus buildings in a rapidly changing UK climate. This paper sets out to investigate whether Passivhaus dwellings will be able to provide high standards of thermal comfort in the future or whether they are inherently vulnerable to overheating risks. Scenario modelling using probabilistic data derived from the UKCP09 weather generator (WG) in conjunction with dynamic simulation and global sensitivity analysis techniques are used to assess the future performance of a range of typical Passivhaus dwellings relative to an identical Fabric Energy Efficiency Standard (FEES) compliant dwelling over its notional future lifespan. The emphasis of this study is to understand what impact climate change will pose to overheating risks for Passivhaus dwellings relative to the de facto (i.e. FEES) alternative, and which design factors play a dominant role in contributing to this risk. The results show that optimization of a small number of design inputs, including glazing ratios and external shading devices, can play a significant role in mitigating future overheating risks.

A new variance-based global sensitivity analysis technique

A new set of variance-based sensitivity indices, called W-indices, is proposed. Similar to the Sobol’s indices, both main and total effect indices are defined. The W-main effect indices measure the average reduction of model output variance when the ranges of a set of inputs are reduced, and the total effect indices quantify the average residual variance when the ranges of the remaining inputs are reduced. Geometrical interpretations show that the W-indices gather the full information of the variance ratio function, whereas, Sobol’s indices only reflect the marginal information. Then the double-loop-repeated-set Monte Carlo (MC) (denoted as DLRS MC) procedure, the double-loop-single-set MC (denoted as DLSS MC) procedure and the model emulation procedure are introduced for estimating the W-indices. It is shown that the DLRS MC procedure is suitable for computing all the W-indices despite its highly computational cost. The DLSS MC procedure is computationally efficient, however, it is only applicable for computing low order indices. The model emulation is able to estimate all the W-indices with low computational cost as long as the model behavior is correctly captured by the emulator. The Ishigami function, a modified Sobol’s function and two engineering models are utilized for comparing the W- and Sobol’s indices and verifying the efficiency and convergence of the three numerical methods. Results show that, for even an additive model, the W-total effect index of one input may be significantly larger than its W-main effect index. This indicates that there may exist interaction effects among the inputs of an additive model when their distribution ranges are reduced.

Global sensitivity analysis of hydrologic processes in major snow‐dominated mountainous river basins in Colorado

AbstractThe performance of watershed models in simulating stream discharge depends on the adequate representation of important watershed processes. In snow‐dominated systems, snow, surface and subsurface hydrologic processes comprise a complex network of nonlinear interactions that influence the magnitude and timing of discharge. This study aims to identify critical processes and interactions that control discharge hydrographs in five major mountainous snow‐dominated river basins in Colorado, USA. A comprehensive watershed model (Soil and Water Assessment Tool) and a variance‐based global sensitivity analysis technique (Fourier Amplitude Sensitivity Test) were used in conjunction to identify critical models parameters and processes that they represent. Average monthly streamflow and streamflow root mean square error over a period of 20 years were used as two separate objective functions in this analysis. Examination of the sensitivity of monthly streamflow revealed the influence of parameters on flow volume, whereas the sensitivity of streamflow root mean square error also exposed the influence of parameters on the timing of the hydrographs. A stability analysis was performed to investigate the computational requirements for a robust sensitivity analysis. Results show that streamflow volume is mostly influenced by shallow subsurface processes, whereas interactions between groundwater and snow processes were the key in the timing of streamflows. A large majority of important parameters were common among all study watersheds, which underlies the prospect for regionalization of process‐based hydrologic modelling in headwater river basins in Colorado. Copyright © 2013 John Wiley & Sons, Ltd.

Assessment of the contamination of drinking water supply wells by pesticides from surface water resources using a finite element reactive transport model and global sensitivity analysis techniques

► Surface water contaminants can reach water wells located in confined aquifers. ► Global sensitivity analysis tools allowed identifying most important parameters. ► Well depth, pesticide characteristics and clay layer thickness are very important. ► Persistent compounds in surface water can pose a threat to drinking water supply wells. A reactive transport model is employed to evaluate the potential for contamination of drinking water wells by surface water pollution. The model considers various geologic settings, includes sorption and degradation processes and is tested by comparison with data from a tracer experiment where fluorescein dye injected in a river is monitored at nearby drinking water wells. Three compounds were considered: an older pesticide MCPP (Mecoprop) which is mobile and relatively persistent, glyphosate (Roundup), a newer biodegradable and strongly sorbing pesticide, and its degradation product AMPA. Global sensitivity analysis using the Morris method is employed to identify the dominant model parameters. Results show that the characteristics of clay aquitards (degree of fracturing and thickness), pollutant properties and well depths are crucial factors when evaluating the risk of drinking water well contamination from surface water. This study suggests that it is unlikely that glyphosate in streams can pose a threat to drinking water wells, while MCPP in surface water can represent a risk: MCPP concentration at the drinking water well can be up to 7% of surface water concentration in confined aquifers and up to 10% in unconfined aquifers. Thus, the presence of confining clay aquitards may not prevent contamination of drinking water wells by persistent compounds in surface water. Results are consistent with data on pesticide occurrence in Denmark where pesticides are found at higher concentrations at shallow depths and close to streams.

Necessary condition for applying experimental design criteria to global sensitivity analysis results

Conventional techniques for optimal experimental design are based on local sensitivity analysis to quantify parameter effects on the output. However, one of the key challenges for experimental design is that the local sensitivity is dependent on the unknown parameter values for a nonlinear model. This problem can be addressed if the sensitivity matrix, used in experimental design, could be computed by global sensitivity analysis techniques rather than local sensitivity analysis methods. However, not all existing global sensitivity analysis measures can compute such a sensitivity matrix. This paper presents a necessary condition for integrating global sensitivity analysis with experimental design criteria, i.e., the design criterion of the global sensitivity matrix reduces to the one applied to the local sensitivity matrix if the parameter uncertainty range tends to zero. Four different sensitivity measures are analyzed using this condition and the results are illustrated in a detailed case study where a comparison with local design and Bayesian design is made.

Global sensitivity analyses for a complex hydrological model applied in an Alpine watershed

AbstractThe increasing complexity of hydrological models results in a large number of parameters to be estimated. In order to better understand how these complex models work, efficient screening methods are required in order to identify the most important parameters. This is of particular importance for models that are used within an operational real‐time forecasting chain such as HQsim. The objectives of this investigation are to (i) identify the most sensitive parameters of the complex HQsim model applied in the Alpine Lech catchment and (ii) compare model parameter sensitivity rankings attained from three global sensitivity analysis techniques. The techniques presented are the (i) regional sensitivity analysis, (ii) Morris analysis and (iii) state‐dependent parameter modelling. The results indicate that parameters affecting snow melt as well as processes in the unsaturated soil zone reveal high significance in the analysed catchment. The snow melt parameters show clear temporal patterns in the sensitivity whereas most of the parameters affecting processes in the unsaturated soil zone do not vary in importance across the year. Overall, the maximum degree day factor (meltfunc_max) has been identified to play a key role within the HQsim model. Although the parameter sensitivity rankings are equivalent between methods for a number of parameters, for several key parameters differing results were obtained. An uncertainty analysis demonstrates that a parameter ranking attained from only one method is subjected to large uncertainty. Copyright © 2012 John Wiley & Sons, Ltd.

Estimating Sobol sensitivity indices using correlations

Sensitivity analysis is a crucial tool in the development and evaluation of complex mathematical models. Sobol's method is a variance-based global sensitivity analysis technique that has been applied to computational models to assess the relative importance of input parameters on the output. This paper introduces new notation that describes the Sobol indices in terms of the Pearson correlation of outputs from pairs of runs, and introduces correction terms to remove some of the spurious correlation. A variety of estimation techniques are compared for accuracy and precision using the G function as a test case.

A novel global sensitivity analysis approach in testing measurement algorithms used by protective relays

SUMMARY Measurement algorithm is the important element of Intelligent Electronic Devices (IEDs) used in protection and monitoring of transmission and distribution networks. The function of the measurement algorithm is to estimate fundamental frequency component (phasor) of current and voltage signals recorded during network faults. The estimated fundamental component is used by variety of protection algorithms for the successful operation of the IEDs. The most popular measurement algorithm is the Discrete Fourier Transform (DFT). However, the estimation of the fundamental frequency component by the DFT algorithm will produce error if input signal contains nuisance components such as a decaying DC offset. This paper evaluates the performance of DFT measurement algorithm when its input signal is influenced by variety of the nuisance components. Various random factors will impact the amount of nuisance components present in a measured signal. Therefore, we propose a novel global uncertainty and sensitivity analysis technique to determine in a systematic way the impact of nuisance components on the performance of measurement algorithms. In addition to the full-cycle DFT, we analyze performance of half-cycle DFT and full-cycle cosine filter algorithms. Impact of nuisance components due to current transformer (CT) saturation and without saturation was studied. Steady-state and transient performance indices are defined to evaluate the performance of the algorithms. Copyright © 2012 John Wiley & Sons, Ltd.

Computational Method for Global Sensitivity Analysis of Reactor Neutronic Parameters

The variance-based global sensitivity analysis technique is robust, has a wide range of applicability, and provides accurate sensitivity information for most models. However, it requires input variables to be statistically independent. A modification to this technique that allows one to deal with input variables that are blockwise correlated and normally distributed is presented. The focus of this study is the application of the modified global sensitivity analysis technique to calculations of reactor parameters that are dependent on groupwise neutron cross-sections. The main effort in this work is in establishing a method for a practical numerical calculation of the global sensitivity indices. The implementation of the method involves the calculation of multidimensional integrals, which can be prohibitively expensive to compute. Numerical techniques specifically suited to the evaluation of multidimensional integrals, namely, Monte Carlo and sparse grids methods, are used, and their efficiency is compared. The method is illustrated and tested on a two-group cross-section dependent problem. In all the cases considered, the results obtained with sparse grids achieved much better accuracy while using a significantly smaller number of samples. This aspect is addressed in a ministudy, and a preliminary explanation of the results obtained is given.

Multi-Objective Sensitivity Analysis of a Fully Distributed Hydrologic Model WetSpa

The application of fully distributed watershed models has the advantage of providing location-specific outputs. However, the calibration of these models is very challenging due to over-parameterization. A typical strategy is to aggregate parameters and screen out insensitive parameters in order to decrease the dimension of the problem for calibration. To ensure the validity of calibration, it is important to identify important physical processes and parameter interactions, and examine how different model setups affect model simulation. In this paper, a two-step multi-objective sensitivity analysis approach is applied to a distributed hydrologic model, the WetSpa (Water and Energy Transfer between Soil, Plant and Atmosphere), with case studies in the Chaohe Basin in China and the Margecany Basin in Slovakia respectively. This two-step global sensitivity analysis technique, incorporating the Morris method and the SDP (State Dependent Parameter) method, has proved to be effective in the two case studies. The results of two case studies show that (i) a warm-up period is essential for minimizing the impact of initial state variables to the model simulation, (ii) different objective functions lead to different sensitivity results, (iii) evapotranspiration is the most sensitive process to the model result in the two study watersheds followed by the groundwater and soil water process, and (iv) the sensitivity of snowmelt process is case dependent.

Biological reduction of chlorinated solvents: Batch-scale geochemical modeling

Simulation of biodegradation of chlorinated solvents in dense non-aqueous phase liquid (DNAPL) source zones requires a model that accounts for the complexity of processes involved and that is consistent with available laboratory studies. This paper describes such a comprehensive modeling framework that includes microbially mediated degradation processes, microbial population growth and decay, geochemical reactions, as well as interphase mass transfer processes such as DNAPL dissolution, gas formation and mineral precipitation/dissolution. All these processes can be in equilibrium or kinetically controlled. A batch modeling example was presented where the degradation of trichloroethene (TCE) and its byproducts and concomitant reactions (e.g., electron donor fermentation, sulfate reduction, pH buffering by calcite dissolution) were simulated. Local and global sensitivity analysis techniques were applied to delineate the dominant model parameters and processes. Sensitivity analysis indicated that accurate values for parameters related to dichloroethene (DCE) and vinyl chloride (VC) degradation (i.e., DCE and VC maximum utilization rates, yield due to DCE utilization, decay rate for DCE/VC dechlorinators) are important for prediction of the overall dechlorination time. These parameters influence the maximum growth rate of the DCE and VC dechlorinating microorganisms and, thus, the time required for a small initial population to reach a sufficient concentration to significantly affect the overall rate of dechlorination. Self-inhibition of chlorinated ethenes at high concentrations and natural buffering provided by the sediment were also shown to significantly influence the dechlorination time. Furthermore, the analysis indicated that the rates of the competing, nonchlorinated electron-accepting processes relative to the dechlorination kinetics also affect the overall dechlorination time. Results demonstrated that the model developed is a flexible research tool that is able to provide valuable insight into the fundamental processes and their complex interactions during bioremediation of chlorinated ethenes in DNAPL source zones.

Quantitative Optimal Experimental Design Using Global Sensitivity Analysis via Quasi-Linearization

Local sensitivity analysis is widely used in experimental design to improve the precision of the estimated parameters. However, for nonlinear models, the local sensitivity values and the experimental design criteria are dependent on the, not yet known, parameter values. Global sensitivity analysis can deal with this situation by taking parameter uncertainty into account for computation of the sensitivity values. However, the existing experimental design criteria cannot easily be applied to the conventional global sensitivity analysis results. One outcome of this is that experimental design involving global sensitivity analysis has mainly focused on identification of influential parameters. A new global sensitivity analysis technique is presented in this work for the purpose of using this technique for quantitative experimental design. The methodology makes use of quasi-linearization and the global sensitivity matrix returned is the design matrix of the linearized model. Due to this, the same experimental criteria that have been developed for quantitative optimal design of linear models can be applied and serve as indicators of desired properties of the parameter estimates. The presented design using global sensitivity analysis is consistent with the popular design involving local sensitivity analysis when the parameter uncertainty is small; however, the technique outperforms local design when applied to models with significant parameter uncertainty.

Sensitivity and identifiability of stream flow generation parameters of the SWAT model

AbstractImplementation of sensitivity analysis (SA) procedures is helpful in calibration of models and also for their transposition to different watersheds. The reported studies on SA of Soil and Water Assessment Tool (SWAT) model were mostly focused on identifying parameters for pruning or modifying during the calibration process. This paper presents a sensitivity and identifiability analysis of model parameters that influence stream flow generation in SWAT. The analysis was focused on evaluating the sensitivity of the parameters in different climatic settings, temporal scales and flow regimes. The global sensitivity analysis (GSA) technique based on classical decomposition of variance, Sobol', was employed in this study. The results of the study indicate that modeled stream flow show varying sensitivity to parameters in different climatic settings. The results also suggest that the identifiability of a parameter for a given watershed is a major concern in calibrating the model for the specific watershed, as it might lead to equifinality of parameters. The SWAT model parameters show varying sensitivity in different years of simulation suggesting the requirement for dynamic updation of parameters during the simulation. The sensitivity of parameters during various flow regimes (low, medium and high flow) is also found to be uneven, which suggests the significance of a multi‐criteria approach for the calibration of models. Copyright © 2010 John Wiley & Sons, Ltd.