Sobol Methods Research Articles

Robust combination of the Morris and Sobol methods in complex multidimensional models

Conducting global sensitivity analysis using variance decomposition methods in complex simulation models with many input factors is usually unaffordable. An alternative is to first apply a screening method to reduce the number of input factors and then apply a variance decomposition method to the reduced model. However, usually selection of input factors is not done robustly and convergence of the screening method is not ensured.We propose two new criteria, a criterion that mimics the visual selection of the input factors and a convergence criterion. In the application of the criteria to a complex model, the Morris screening method has needed 200 trajectories to converge and the visual criterion has outperformed other existing criteria. Our proposal ensures a robust combination of the Morris and the Sobol methods that provides an objective and automatic method to select the most important input factors with a feasible computing load to achieve convergence.

Extending sensitivity analysis using regression to effectively disseminate life cycle assessment results

Bottom-up–based life cycle assessment (LCA) approaches are used to assess the greenhouse gas emissions of various products such as transportation fuels. Bottom-up spreadsheet–based models include numerous calculations and assumed values that are uncertain. Currently, most LCAs provide point estimates with a simple one-at-a-time sensitivity analysis, which provides limited insight into how the model assumptions affect the results. Additionally, the LCA models are generally presented with a limited number of scenarios to avoid overwhelming the reader; however, this limits the usefulness of the work, as each reader will be interested in different scenarios. The goal of this work is to use a global sensitivity and regression to provide as much information to the reader as possible in an easily digestible form. The Morris and Sobol global sensitivity methods are examined to determine if they can accurately identify the key inputs that have the largest effect on overall output variance. A multiparameter linear regression is then used to simplify the model into a single equation. Rstudio and Excel VBA are used to create an easy-to-use template called the Regression, Uncertainty, and Sensitivity Tool (RUST) that can be inserted into any Excel-based LCA model. This method is applied to the previously published FUNdamental ENgineering PrinciplEs-based ModeL for Estimation of GreenHouse Gases in Conventional Crude Oils and Oil Sands (FUNNEL-GHG-CCO/OS) as an example case. Both the Morris and Sobol methods can identify the key parameters, but the Morris method requires less than 1/100th as many model evaluations. Of the model’s 65 parameters, 14 key parameters were identified. The corresponding regression model was found to have an accuracy of ± 0.5 g CO2 eq/MJ 90% of the time and a maximum error of + 3 and − 1 g CO2 eq/MJ. This work found that the Morris method can be used to screen key parameters and that a stepwise multiparameter linear regression approach can be used to develop a simplified version of the model. The developed RUST Excel workbook can be used to perform the sensitivity and regression analysis of any Excel-based LCA models. The regression model can then be easily published, it does not require a large effort to make a user friendly version of the model, and it conceals confidential data if necessary. The simplified model makes it easy for policy markets to investigate how changes in critical parameters affect the LCA results without having to learn how to use the full complex model.

Approximating concept stability using variance reduction techniques

Finding actionable patterns from massive data is a crucial task in data mining applications. In Formal Concept Analysis (FCA), concept stability is one of the commonly used measures for assessing the interestingness of formal concepts, and hence selecting the relevant patterns of such a type. Accurate and scalable computation of concept stability often remains a key challenge. While exact methods for computing stability can be an effective solution in small applications, their algorithmic complexity is however at least quadratic in the size of the input lattice which could be exponential with respect to the size of the context. As such, approximation algorithms like Monte Carlo Sampling (MCS) have been applied to be more effective in practice. However, MCS often has a slow convergence problem with an inaccurate estimation of stability. In this paper, we introduce a new set of approximation methods to estimate the stability index based on variance reduction techniques. Specifically, we focus on adapting Latin hypercube (LHS), scrambled Sobol (SoS) and Latin supercube (LSS) sampling methods to exploit the potential of stratification and Low-discrepancy as well as hybridization approaches to improve the convergence rate. In contrast to the pure randomness of MCS, the proposed methods aim to spread the sample points more evenly across all possible subsets of concept intent (or extent). This allows all the areas of the intent powerset space to be properly represented. Our experiments on several formal contexts illustrate the efficiency of LHS, SoS and LSS over MCS.

GLOBAL SENSITIVITY ANALYSIS METHODS APPLIED TO HYDROLOGIC MODELING WITH THE SAC-SMA MODEL

ABSTRACT This study aimed to use global sensitivity analysis (GSA) methods to evaluate the sensitivity of the SAC-SMA hydrologic model parameters in the estimation of daily flows of the Piracicaba river basin. The study was carried out in three sections of flow monitoring of the Piracicaba river basin, with an area of 5,304.0 km2 and located in the State of Minas Gerais, Brazil. For the global sensitivity analysis of the SAC-SMA model, the Morris and Sobol methods were used. The model parameters showing high sensitivity were UZFWM, which represents the free water depth in the upper zone of the soil and interferes with the subsurface flow and groundwater aquifer recharge; ADIMP, which represents the additional impermeable area of the basin and interferes with the direct runoff generation; and LZPK, LZSK, LZFPM, and LZFSM, which are related to the base flow of the basin. The results showed that most of the SAC-SMA model parameters do not provide expressive variations in the output variable, most of the SAC-SMA model parameters with high sensitivity are the intervenient in the base flow, and the Morris method should be used as a preliminary analysis to the use of the Sobol method.

Fast sensitivity analysis methods for computationally expensive models with multi-dimensional output

Abstract. Global sensitivity analysis (GSA) is a powerful approach in identifying which inputs or parameters most affect a model's output. This determines which inputs to include when performing model calibration or uncertainty analysis. GSA allows quantification of the sensitivity index (SI) of a particular input – the percentage of the total variability in the output attributed to the changes in that input – by averaging over the other inputs rather than fixing them at specific values. Traditional methods of computing the SIs using the Sobol and extended Fourier Amplitude Sensitivity Test (eFAST) methods involve running a model thousands of times, but this may not be feasible for computationally expensive Earth system models. GSA methods that use a statistical emulator in place of the expensive model are popular, as they require far fewer model runs. We performed an eight-input GSA, using the Sobol and eFAST methods, on two computationally expensive atmospheric chemical transport models using emulators that were trained with 80 runs of the models. We considered two methods to further reduce the computational cost of GSA: (1) a dimension reduction approach and (2) an emulator-free approach. When the output of a model is multi-dimensional, it is common practice to build a separate emulator for each dimension of the output space. Here, we used principal component analysis (PCA) to reduce the output dimension, built an emulator for each of the transformed outputs, and then computed SIs of the reconstructed output using the Sobol method. We considered the global distribution of the annual column mean lifetime of atmospheric methane, which requires ∼ 2000 emulators without PCA but only 5–40 emulators with PCA. We also applied an emulator-free method using a generalised additive model (GAM) to estimate the SIs using only the training runs. Compared to the emulator-only methods, the emulator–PCA and GAM methods accurately estimated the SIs of the ∼ 2000 methane lifetime outputs but were on average 24 and 37 times faster, respectively.

Time-dependent global sensitivity analysis of the C-RIVE biogeochemical model in contrasted hydrological and trophic contexts

Dissolved oxygen within water column is a key variable to characterize the water quality. Water quality modeling has been extensively developed for decades. However, complex biogeochemical cycles are described using a high number of parameters. Hence, parameters' uncertainty constitutes a major problem in the application of these models. Sensitivity analysis allows the identification of the most influential parameters in a model and a better understanding of the governing processes. This paper presents a time-dependent sensitivity analysis for dissolved oxygen using Morris and Sobol methods combined with a functional principal components analysis for dimension reduction. The aim of this study is to identify the most important parameters of C-RIVE model in different trophic contexts and to understand the biogeochemical functioning of river systems. The results indicate that the maintenance respiration of phytoplankton and the photosynthetic parameters (i.e. photosynthetic capacity, the maximal photosynthesis rate and light extinction coefficients) are the most influential parameters during algal blooms. When the river system becomes heterotrophic, the bacterial activities (moderate and high temperature) and the reaeration coefficients (low temperature) affect the most the dissolved oxygen concentration in the water column. An anthropogenic effect (ship navigation) on variation of dissolved oxygen concentration has been identified and the role of this anthropogenic effect evolves with hydrological and trophic conditions.

A new framework for comprehensive, robust, and efficient global sensitivity analysis: 1. Theory

AbstractComputer simulation models are continually growing in complexity with increasingly more factors to be identified. Sensitivity Analysis (SA) provides an essential means for understanding the role and importance of these factors in producing model responses. However, conventional approaches to SA suffer from (1) an ambiguous characterization of sensitivity, and (2) poor computational efficiency, particularly as the problem dimension grows. Here, we present a new and general sensitivity analysis framework (called VARS), based on an analogy to “variogram analysis,” that provides an intuitive and comprehensive characterization of sensitivity across the full spectrum of scales in the factor space. We prove, theoretically, that Morris (derivative‐based) and Sobol (variance‐based) methods and their extensions are special cases of VARS, and that their SA indices can be computed as by‐products of the VARS framework. Synthetic functions that resemble actual model response surfaces are used to illustrate the concepts, and show VARS to be as much as two orders of magnitude more computationally efficient than the state‐of‐the‐art Sobol approach. In a companion paper, we propose a practical implementation strategy, and demonstrate the effectiveness, efficiency, and reliability (robustness) of the VARS framework on real‐data case studies.

Numerical analysis method of heat transfer in an electronic component using sensitivity analysis

Numerical models applied to real-life engineering processes have many input parameters that are not always known with a sufficient degree of certainty. Moreover, models are characterized by their mathematical complexity and time-consuming calculations. In this context, our certified reduced basis (crb) model is developed taking into account the input uncertainties which produce many uncertain and different outputs. In order to assess more precisely the relative importance of input parameters on the model outputs, we complete and use a quantitative sensitivity analysis which is a crucial tool in the development and evaluation of complex models. In the current paper, the parametrized crb method is introduced and applied successfully for a simplified 2D heat transfer of electronic components. Furthermore, different sensitivity analysis methods have been compared; Specifically, we have used linear sensitivity methods, one factor at a time design, and global techniques such as Morris one factor at a time, Sobol's methods and Monte Carlo sampling coupled with scatter plot and correlation analysis.

The use of global sensitivity analysis for improving processes: Applications to mineral processing

This paper analyzes the application of global sensitivity analysis (GSA) to the improvement of processes using various case studies. First, a brief description of the methods applied is given, and several case studies are examined to show how GSA can be applied to the study to improve the processes. The case studies include the identification of processes; comparisons of the Sobol, E-FAST and Morris GSA methods; a comparison of GSA with local sensitivity analysis; an examination of the effect of uncertainty levels and the type of distribution function on the input factors; and the application of GSA to the improvement of a copper flotation circuit. We conclude that GSA can be a useful tool in the analysis, comparison, design and characterization of separation circuits. In addition, we conclude that using the stage's recoveries of each species as input factors is a suitable choice for the GSA of a flotation plant.

Addressing ten questions about conceptual rainfall–runoff models with global sensitivity analyses in R

Summary Sensitivity analysis (SA) is generally recognized as a worthwhile step to diagnose and remedy difficulties in identifying model parameters, and indeed in discriminating between model structures. An analysis of papers in three journals indicates that SA is a standard omission in hydrological modeling exercises. We provide some answers to ten reasonably generic questions using the Morris and Sobol SA methods, including to what extent sensitivities are dependent on parameter ranges selected, length of data period, catchment response type, model structures assumed and climatic forcing. Results presented demonstrate the sensitivity of four target functions to parameter variations of four rainfall–runoff models of varying complexity (4–13 parameters). Daily rainfall, streamflow and pan evaporation data are used from four 10-year data sets and from five catchments in the Australian Capital Territory (ACT) region. Similar results are obtained using the Morris and Sobol methods. It is shown how modelers can easily identify parameters that are insensitive, and how they might improve identifiability. Using a more complex objective function, however, may not result in all parameters becoming sensitive. Crucially, the results of the SA can be influenced by the parameter ranges selected. The length of data period required to characterize the sensitivities assuredly is a minimum of five years. The results confirm that only the simpler models have well-identified parameters, but parameter sensitivities vary between catchments. Answering these ten questions in other case studies is relatively easy using freely available software with the Hydromad and Sensitivity packages in R.

Portfolio Optimization for VAR, CVaR, Omega and Utility with General Return Distributions: A Monte Carlo Approach for Long-Only and Bounded Short Portfolios with Optional Robustness and a Simplified Approach to Covariance Matching

We develop the idea of using Monte Carlo sampling of random portfolios to solve portfolio investment problems. We explore the need for more general optimization tools, and consider the means by which constrained random portfolios may be generated. DeVroye's approach to sampling the interior of a simplex (a collection of non-negative random variables adding to unity) is already available for interior solutions of simple fully-invested long-only systems, and we extend this to treat, lower bound constraints, bounded short positions and to sample non-interior points by the of Face-Edge-Vertex-biased sampling. A practical scheme for long-only and bounded short problems is developed and tested. Non-convex and disconnected regions can be treated by applying rejection for other constraints. The advantage of Monte Carlo methods is that they may be extended to risk functions that are more complicated functions of the return distribution, without explicit gradients, and that the underlying return distribution may be modeled parametrically or empirically based on general distributions. The optimization of expected utility, Omega, Sortino ratios may be handled in a similar manner to quadratic risk, VaR and CVaR, irrespective of whether a reduction to LP or QP form is available. Robustification is also possible, and a Monte Carlo approach allows the possibility of relaxing the general maxi-min approach to one of varying degrees of conservatism. Grid computing technology is an excellent platform for the development of such computations due to the intrinsically parallel nature of the computation. Good comparisons with established results in Mean-Variance and CVaR optimization are obtained, and we give some applications to Omega and expected Utility optimization. Extensions to deploy Sobol and Niederreiter quasi-random methods for random weights are also proposed. Extensions to the value functions of prospect theory are possible. The initial proposed here is essentially an initial global search which produces a good feasible solution for any number of assets with any risk function and return distribution. This solution is close to optimal in lower dimensions. A by-product of these investigations is a simplified approach - the double-Cholesky method - to sampling certain multivariate distributions matching the covariance matrix as well as the mean vector.

Modern sensitivity analysis of the CESARE-Risk computer fire model

This paper introduces two new modern methods of global sensitivity analysis for computer models: Fourier Amplitude and Sobol, as well as a modern factor screening method: the Morris method. The methods are applied to the sensitivity analysis of the apartment fire module of the CESARE-Risk building fire computer model with eight input factors and door and window open. Two output variables are considered: the maximum temperature reached and the time of onset of untenable conditions. Response surfaces previously derived for the model [Jianguo Qu, Response surface modelling of Monte Carlo fire data, Ph.D. Thesis, Victoria University, Melbourne, Australia, 2003, http://eprints.vu.edu.au/archive/00000260/01/Qu,_Jianguo.pdf] are used to speed up the computations. For the maximum temperature all three methods agree that the most sensitive factors are the window height and width factors, followed by the fuel area factor. The largest interaction was between the length of room and the fuel area factor. For the time of untenable conditions the Fourier Amplitude and Sobol methods agreed that one factor, the flame spread rate, had overwhelming significance. The only significant Sobol interaction was between the width of room and the flame spread rate.