Model Output Variables Research Articles

Risk of estimators for Sobol’ sensitivity indices based on metamodels

Sobol' sensitivity indices allow to quantify the respective effects of random input variables and their combinations on the variance of mathematical model output. We focus on the problem of Sobol' indices estimation via a metamodeling approach where we replace the true mathematical model with a sample-based approximation to compute sensitivity indices. We propose a new method for indices quality control and obtain asymptotic and non-asymptotic risk bounds for Sobol' indices estimates based on a general class of metamodels. Our analysis is closely connected with the problem of nonparametric function fitting using the orthogonal system of functions in the random design setting. It considers the relation between the metamodel quality and the error of the corresponding estimator for Sobol' indices and shows the possibility of fast convergence rates in the case of noiseless observations. The theoretical results are complemented with numerical experiments for the approximations based on multivariate Legendre and Trigonometric polynomials.

Soft computing-based modeling and emission control/reduction of a diesel engine fueled with carbon nanoparticle-dosed water/diesel ‎emulsion fuel

This study was set up to model and optimize the performance and emission characteristics of a diesel engine fueled with carbon nanoparticle-dosed water/‎diesel emulsion fuel using a combination of soft computing techniques. Adaptive neuro-fuzzy inference system tuned by particle ‎swarm algorithm was used for modeling the performance and emission parameters of the engine, while optimization of the engine operating parameters and the fuel composition was conducted via multiple-objective particle ‎swarm algorithm. The model input variables were: injection timing (35–41° CA BTDC), engine load (0–100%), nanoparticle dosage (0–150 μM), and water content (0–3 wt%). The model output variables included: brake specific fuel consumption, brake thermal efficiency, as well as carbon monoxide, carbon dioxide, nitrogen oxides, and unburned hydrocarbons emission concentrations. The training and testing of the modeling system were performed on the basis of 60 data patterns obtained from the experimental trials. The effects of input variables on the performance and emission characteristics of the engine were thoroughly analyzed and comprehensively discussed as well. According to the experimental results, injection timing and engine load could significantly affect all the investigated performance and emission parameters. Water and nanoparticle addition to diesel could markedly affect some performance and emission parameters. The modeling system could predict the output parameters with an R2 > 0.93, MSE < 5.70 × 10−3, RMSE < 7.55 × 10−2, and MAPE < 3.86 × 10−2. The optimum conditions were: injection timing of 39° CA BTDC, engine load of 74%, nanoparticle dosage of 112 μM, and water content of 2.49 wt%. The carbon dioxide, carbon monoxide, nitrogen oxides, and unburned hydrocarbon emission concentrations ‎were found to be ‎7.26‎ vol%‎, ‎0.46 vol%‎, ‎95.7‎ ppm, and‎ 36.2 ppm, respectively, under the ‎selected optimal operating conditions while the quantity of brake thermal efficiency was found at an acceptable level (‎34.0‎%).‎ In general, the applied soft computing combination appears to be a promising approach to model and optimize operating parameters and fuel composition of diesel engines.

A Novel Ensemble Design for Probabilistic Predictions of Fine Particulate Matter Over the Contiguous United States (CONUS)

AbstractThis study examines the benefit of using a dynamical ensemble for 48 hr deterministic and probabilistic predictions of near‐surface fine particulate matter (PM2.5) over the contiguous United States (CONUS). Our ensemble design captures three key sources of uncertainties in PM2.5 modeling including meteorology, emissions, and secondary organic aerosol (SOA) formation. Twenty‐four ensemble members were simulated using the Community Multiscale Air Quality (CMAQ) model during January, April, July, and October 2016. The raw ensemble mean performed better than most of the ensemble members but underestimated the observed PM2.5 over the CONUS with the largest underestimation over the western CONUS owing to negative PM2.5 bias in nearly all the members. To improve the ensemble performance, we calibrated the raw ensemble using model output statistics (MOS) and variance deficit methods. The calibrated ensemble captured the diurnal and day‐to‐day variability in observed PM2.5 very well and exhibited almost zero mean bias. The mean bias in the calibrated ensemble was reduced by 90–100% in the western CONUS and by 40–100% in other parts of the CONUS, compared to the raw ensemble in all months. The corresponding reduction in root‐mean‐square error (RMSE) was 13–40%. The calibrated ensemble also showed 30% improvement in the RMSE and spread matching compared to the raw ensemble. We have also shown that a nine‐member ensemble based on combinations of three meteorological and three anthropogenic emission scenarios can be used as a smaller subset of the full ensemble when sufficient computational resources are not available in the operational setting.

Sensitivity analysis on a three-phase plant-wide water and resource recovery facility model for identification of significant parameters

Water and resource recovery facility (WRRF) mathematical models have been advancing towards their widespread application for sizing and operation of treatment plants to minimize energy consumption and cost while maximizing nutrient recovery and effluent quality. Effective utilisation of these models requires that they are well calibrated. However, difficulties (with important parameters not identified and uncertainties in interpretation of model output results) can be experienced in model calibration, especially due to (i) the intricate relationships of model output variables with model input factors (where parameters are inter-related to various model outputs), resulting in non-linearity, and (ii) the limitations (due to expensive and/or time-consuming experimental methods) experienced in procuring and reconciling data required for determination of the model input factors. This paper presents the performance of a sensitivity analysis, reinforced with expert-based reasoning, on a three-phase (aqueous-gas-solid) plant-wide model (PWM_SA, Ikumi et al., 2015), for identification of significant parameters, and highlights the ones requiring experimental determination, specific to the system. The sensitivity analysis exercise was performed using two methods – i.e., Morris screening (screening method) and standardised regression coefficient (SRC; based on regression). This process was useful towards detection of the parameters, which are not normally measured at WRRFs, but may require attention for future application of mathematical models in decision-making processes for WRRFs. These included the influent fractions of unbiodegradable and readily biodegradable organics, the kinetic constants for hydrolysis of biodegradable particulates, the elemental composition of the organics and the specific growth rate of autotrophic nitrifying biomass.

Cost-effectiveness of surgery for drug-resistant temporal lobe epilepsy in the US.

Surgery is an effective but costly treatment for many patients with drug-resistant temporal lobe epilepsy (DR-TLE). We aim to evaluate whether, in the United States, surgery is cost-effective compared to medical management for patients deemed surgical candidates and whether surgical evaluation is cost-effective for patients with DR-TLE in general. We use a semi-Markov model to assess the cost-effectiveness of surgery and surgical evaluation over a lifetime horizon. We use second-order Monte Carlo simulations to conduct probabilistic sensitivity analyses to estimate variation in model output. We adopt both health care and societal perspectives, including direct health care costs (e.g., surgery, antiepileptic drugs) and indirect costs (e.g., lost earnings by patients and care providers.) We compare the incremental cost-effectiveness ratio to societal willingness to pay (∼$100,000 per quality-adjusted life-year [QALY]) to determine whether surgery is cost-effective. Epilepsy surgery is cost-effective compared to medical management in surgically eligible patients by virtue of being cost-saving ($328,000 vs $423,000) and more effective (16.6 vs 13.6 QALY) than medical management in the long run. Surgical evaluation is cost-effective in patients with DR-TLE even if the probability of being deemed a surgical candidate is only 5%. From a societal perspective, surgery becomes cost-effective within 3 years, and 89% of simulations favor surgery over the lifetime horizon. For surgically eligible patients with DR-TLE, surgery is cost-effective. For patients with DR-TLE in general, referral for surgical evaluation (and possible subsequent surgery) is cost-effective. Patients with DR-TLE should be referred for surgical evaluation without hesitation on cost-effectiveness grounds.

Assessing cloudiness effect on soybean yield in the Southeast Brazil

The solar radiation is one of the most important weather variables for determining the potential yield of agricultural crops. Soybean is the main Brazilian agricultural commodity, with great social and economical importance for the country. It is well recognized that cloudiness is a limiting factor for crop growth rate. Few studies have been conducted to systematically evaluate how much cloud affect soybean yield and none, to our knowledge, is available for tropical soybean. The objective of this paper was to quantify the implications of cloudiness on soybean growth and development in tropical Brazil. To do so, experimental data associated with the simulations of the DSSAT/CROPGRO (DC) model was used, and two treatments were simulated: a) the first used measured solar radiation and b) the second used estimated solar radiation for non-cloud days. Thus, based on the model output variables (growth rate of aerial dry matter, grain yield, specific leaf weight, and light saturation point) and comparing the data with the literature review. We found that simulations exhibited an increase in the dry matter production rate of up to 23%, during days of clear sky, resulting in a yield increase between 26 and 37%. Also, the DC could simulate some adaptation in the plant when the solar radiation changes, like the specific foliar weight and the light-saturated photosynthesis rate.

Derivative-based global sensitivity measure using radial basis function

Global sensitivity analysis (GSA) is always used to measure the contribution of input variables on the variation of model output. In the fields of GSA, derivative-based global sensitivity measure (DGSM) has gained much attention because of its high efficiency in the identification of insignificant input variables. The commonly used approach to estimate DGSM is the Morris method, which requires a large number of model evaluations to compute the gradient of model output. Therefore, it leads to a great challenge in the computational costs, particularly for expensive problems. To alleviate the intensive computation, an estimator based on Gaussian radial basis function (RBF) metamodel is proposed to compute DGSM. With the aid of RBF, the DGSM can be formulated in an explicit expression of one-dimensional Gaussian integrals. For the case of uniformly and normally distributed variables, the analytical expressions of the integral can be derived while for other cases with general distribution, the numerical approach with the quasi-Monte Carlo method is developed to approximate the integrals. The performance of the proposed method is validated by five numerical examples, which illustrate the proposed RBF-based estimator has the capability to yield desirable accuracy and convergence with a limited number of model evaluations.

Making sense of parameter estimation and model simulation in bioprocesses.

Most articles that report fitted parameters for kinetic models do not include meaningful statistical information. This study demonstrates the importance of reporting a complete statistical analysis and shows a methodology to perform it, using functionalities implemented in computational tools. As an example, alginate production is studied in a batch stirred-tank fermenter and modeled using the kinetic model proposed by Klimek and Ollis (1980). The model parameters and their 95% confidence intervals are estimated by nonlinear regression. The significance of the parameters value is checked using a hypothesis test. The uncertainty of the parameters is propagated to the output model variables through prediction intervals, showing that the kinetic model of Klimek and Ollis (1980) can simulate with high certainty the dynamic of the alginate production process. Finally, the results obtained in other studies are compared to show how the lack of statistical analysis can hold back a deeper understanding about bioprocesses.

Multi-Step Approach for Sensitivity Analysis for a Unified Model of Glucose-Insulin Metabolism

Mathematical models of biomedical systems often have a high number of uncertain parameters that are difficult or even impossible to estimate precisely. In order to be able to adequately describe the system, it must be known how large the influence of which factors is on the model behavior and how uncertainties in the parameters affect the model accuracy. Sensitivity analysis (SA) offers a possibility to examine to what extent the variance of the model output can be described by the variability of the input factors. In this paper, a multi-step SA is fulfilled for a unified model of glucose-insulin metabolism that consists of an Elementary Effects Test for screening purposes, a functional principal component analysis for dimensionality reduction of the model output variance and a variance-based approach to determine the sensitivity indices. The concept is tested on several scenarios for type 1 and type 2 diabetic patients, as well as non-diabetics. Results show that parameters are of different importance, depending on the type and scenario studied, which should be considered in a further system analysis.

Application of curve-fitting techniques to develop numerical calibration procedures for a river water quality model

River water quality models are often constrained by a lack of understanding of model structures and complicated estimation procedures for unknown parameters. This paper demonstrates a new calibration strategy by setting up a simple model structure for river water quality. The unknown parameters of RWQM were calibrated through the use of small river water quality data sets. In order to facilitate the calibration procedure, data reconstruction and parameter estimation were performed by the systematic application of cubic smoothing spline, polynomial curve-fitting and nonlinear least squares. The quality of calibrated parameters was estimated by developing a sensitivity ranking system. The variation of model outputs showed a slight difference at a sensitivity index of less than 10% and a significant difference at a sensitivity index of more than 50%. The one-way analysis of variance showed a large p-value of 0.8431, indicating that differences between model data and measured data means are not significant. The calibrated model responses and their statistical envelopes were in good agreement with the river water quality data. A MATLAB GUI platform was developed to perform numerical and graphical analysis, which can be used as a relatively simple but robust calibration tool to support model application and data analysis.

Emulating dynamic non-linear simulators using Gaussian processes

The dynamic emulation of non-linear deterministic computer codes where the output is a time series, possibly multivariate, is examined. Such computer models simulate the evolution of some real-world phenomenon over time, for example models of the climate or the functioning of the human brain. The models we are interested in are highly non-linear and exhibit tipping points, bifurcations and chaotic behaviour. However, each simulation run could be too time-consuming to perform analyses that require many runs, including quantifying the variation in model output with respect to changes in the inputs. Therefore, Gaussian process emulators are used to approximate the output of the code. To do this, the flow map of the system under study is emulated over a short time period. Then, it is used in an iterative way to predict the whole time series. A number of ways are proposed to take into account the uncertainty of inputs to the emulators, after fixed initial conditions, and the correlation between them through the time series. The methodology is illustrated with two examples: the highly non-linear dynamical systems described by the Lorenz and van der Pol equations. In both cases, the predictive performance is relatively high and the measure of uncertainty provided by the method reflects the extent of predictability in each system.

Modelling of food processes under uncertainty: Mechanistic 3D model of chicken meat roasting

Mathematical models that describe the transport phenomena and quality changes of foods during processing contain several uncertain model input parameters which result in uncertain model predictions. The objective of this study was to evaluate the impact of uncertain input parameters on the model predictions of the mechanistic 3D model of chicken meat roasting and to identify as well as rank the most important model parameters. We found that the uncertainty in the model output variables varies with roasting time, but also among the different output variables, by using the Monte Carlo method. To decompose the variance with respect to the input parameters, the method of standardized regression coefficients (SRC), a global sensitivity analysis method, was used. Consequently, the uncertain input parameters were ranked according to their relative impact. The results of the SRC method were compared with the Morris screening, a one-step-at-a-time (OAT) global sensitivity analysis method. The comparison of the two applied sensitivity methods showed that the ranking of the input parameters is similar, while the Morris screening is more efficient computational wise. Finally, we illustrate how the results of the analyses can be used for model refinement as well as to highlight parameters and areas where further research is necessary.

Artificial neural network model for the flow regime recognition in the drying of guava pieces in the spouted bed

In this article, an artificial neural network model (ANN) was developed for the flow regime recognition in the spouted bed dryer. Instabilities and changes were observed in the hydrodynamics of the bed during the drying of guava pieces with deformability and variation in physical properties. Changes in the Archimedes number and Littman parameter directly affected the hydrodynamics of the bed. Experimental data on the variation of the properties of the dried guava pieces were used to obtain the fluid dynamics parameters this was also used as an input data in the ANN model whereas the operating regime of the spouted bed dryer, fixed-,fluidized-,spouted-, and slugging beds were the output model variables. The architecture of the neural network model was selected using the particle swarm optimization algorithm (PSO). The optimized neural model achieved a recognition accuracy of 86% for the fixed and fluidized beds and 99% for the spouted and slugging beds.

Development and Evaluation of Hybrid Artificial Neural Network Architectures for Modeling Spatio-Temporal Groundwater Fluctuations in a Complex Aquifer System

The proper design, development, and appropriate tuning of the Hybrid Neural Network architecture, mainly for its parsimoniousity and optimal training can help practitioners to generate a robust predictive tool for modeling several important hydrological processes within the water resources sector. In this paper, the Feedforward Artificial Neural Network (FFANN) and the hybrid WANN model have been developed, and later, coupled with the Gamma and M-tests (GT) approach for forecasting spatio-temporal groundwater fluctuations in a complex alluvial aquifer system. The performance of these hybrid models were evaluated using goodness-of-fit criteria. An analysis of the modeling results indicates that the GT coupled with the WANN model was able to provide significantly improved results, with lower values of the root mean square error (RMSE) and higher values of the NSE metric for the 1-week and 3-week lead times. Hence, utilizing this hybrid model, the groundwater level prediction tests were extended for 6-week and 12-week lead times with the GT approach, coupled with the WANN hybrid model only. The results showed that the accuracy of the GT-WANN hybrid model was better for the unconfined aquifer system compared to the leaky confined aquifer system. Furthermore, the present study also examined the interdependence between different model inputs and output variables for the selected study sites by means of the Wavelet Coherent Analysis (WCA). These results indicated that all the model’s input variables have a significant effect on the groundwater level of unconfined aquifers, and confirmed the nature of the aquifers tapped within the present study sites. The study finally concludes that the GT-WANN approach can be a robust predictive tool for modeling spatio-temporal fluctuations of groundwater levels.

On the Relative Sensitivity of a Tropical Deep Convective Storm to Changes in Environment and Cloud Microphysical Parameters

Abstract Monte Carlo and Morris screening techniques are used to examine the relative sensitivity of deep convective simulations to changes in initial conditions (IC) versus changes to microphysical parameters (MP). IC are perturbed using a set of empirical orthogonal function–principal component pairs obtained from a database of tropical soundings, while MP are perturbed consistent with their range of realistic values. Monte Carlo experiments provide a broad overview of parameter–output response, while Morris screening techniques identify the most significant influences on specific model output variables. Changes to MP produce a similar order-of-magnitude response in convective hydrologic cycle, dynamics, and latent heating as changes to IC. Changes in IC appear to have a larger effect on radiative fluxes than perturbations to MP. The distribution of low-level latent heating reveals that changes in MP have a larger influence on cold pool properties than changes to the environment. The dominant effects are produced by a subset of cloud MP and thermodynamic structure functions, indicating perturbation of a subset of the control factors may be used to produce most of the variability in a short-term convective-scale ensemble forecast. The most influential MP are the autoconversion threshold, the rain particle size distribution intercept, and the ice particle fall speed parameters. The most influential EOFs are those that correspond to variability in lower- to midtropospheric temperature and water vapor, as well as zonal low-level shear. The results have implications for both the understanding of what influences convective development and the design of ensemble-based prediction and data assimilation systems.

Application of a lower-fidelity surrogate hydraulic model for historic flood reconstruction

Two dimensional hydraulic models are useful to reconstruct maximum discharges and uncertainties of historic flood events. Since many model runs are needed to include the effects of uncertain input parameters, a sophisticated 2D model is not applicable due to computational time. Therefore, this papers studies whether a lower-fidelity model can be used instead. The presented methodological framework shows that a 1D-2D coupled model is capable of simulating maximum discharges with high accuracy in only a fraction of the calculation time needed for the high-fidelity model. Therefore, the lower-fidelity model is used to perform the sensitivity analysis. Multiple Linear Regression analysis and the computation of the Sobol’ indices are used to apportion the model output variance to the most influential input parameters. We used the 1926 flood of the Rhine river as a case study and found that the roughness of grassland areas was by far the most influential parameter.

Ongoing solution reproducibility of earth system models as they progress toward exascale computing

We present a methodology for solution reproducibility for the Energy Exascale Earth System Model during its ongoing software infrastructure development to prepare for exascale computers. The nonlinear chaotic nature of climate system simulations precludes traditional model verification approaches since machine precision differences—resulting from code refactoring, changes in software environment, and so on—grow exponentially to a different weather state. Here, we leverage the nature of climate as a statistical description of the atmosphere in order to establish model reproducibility. We evaluate the degree to which two-sample equality of distribution tests can confidently detect the change in climate from minor tuning parameter changes on model output variables in order to establish the level of difference that indicates a new climate. To apply this (baselined test), we target a section of the model’s development cycle wherein no intentional science changes have been applied to its source code. We compare an ensemble of short simulations that were conducted using a verified model configuration against a new ensemble with the same configuration but with the latest software infrastructure (Common Infrastructure for Modeling the Earth, CIME5.0), compiler versions, and software libraries. We also compare these against ensemble simulations conducted using the original version of the software infrastructure (CIME4.0) of the earlier model configuration, but with the latest compilers and software libraries, to test the impact of new compilers and libraries in isolation from additional software infrastructure. The two-sample equality of distribution tests indicates that these ensembles indeed represent the same climate.

Local and global parameter sensitivity within an ecophysiologically based forest landscape model

Forest landscape models (FLM) are widely used for simulating forest ecosystems. As FLMs have become more mechanistic, more input parameters are required, which increases model parameter uncertainty. To better understand the increased mechanistic detail provided by LANDIS-II/PnET-Succession, we studied the effects of parameter uncertainty on model outputs based on three different approaches. Global sensitivity analyses summarized the influence of each parameter, a local sensitivity analysis determined the magnitude of and degree of nonlinearity of variation in model outputs alongside variation in individual parameters, and a regression tree analysis identified hierarchical relationships among and interaction effects between parameters. Foliar nitrogen, maintenance respiration, and atmospheric carbon dioxide concentration were the most influential parameters in the global analysis. Knowing where parameter influence is concentrated will help model users interpret results from LANDIS-II/PnET-Succession to address ecological questions and should guide priorities for data acquisition.

Consideration of uncertainties in LCA for infrastructure using probabilistic methods

The construction and usage of transport infrastructure are major causes of greenhouse gas emissions and energy consumption. The effects of resource consumption and pollutant emissions are often quantified through Life Cycle Assessment (LCA) models. All decisions made in infrastructure projects during the whole life cycle are afflicted by uncertainty, e.g. physical properties of materials or amount of pollutants emitted by certain processes. The pervasive role of uncertainty is reflected in LCA models, which therefore should consider uncertainty from various sources and provide a sound quantification of their effects. The aim of the work presented in this paper is to give an overview of different sources of uncertainty in LCA of infrastructure projects and to describe systematic methods to evaluate their influence on the results. The possibility of including uncertainty in a LCA-tool for infrastructure is presented, studying the sensitivity of the model output to the input parameters and two alternative approaches for propagation of uncertainty using two case studies. It is shown that, besides the influence of uncertainty in emission factors, other inputs such as material amounts and service life could contribute significantly to the variability of model output and has to be considered if reliable results are sought.

Modeling and energy-based model predictive control of high pressure grinding roll

Even though semiautogenous grinding mills and ball mills are normally used in grinding processes, the industry is driven to decrease cost by increasing efficiencies and decreasing energy consumption. High Pressure Grinding Rolls (HPGR) are seen as an energy-efficient alternative but their developments in modeling and control have received relatively little attention. In this work a model and a control scheme for HPGR is presented that considers the total energy consumed as one of the main controlled variable. First, the model was generated by using literature-reported information of a specific manufacturer and lithology. The dynamic representation of the treatment capacity, product granulometric distribution (reported as 80% percentile), compression energy, and rolling energy were considered as the most important model output variables. Second, model validation was performed with considerable positive results (based on assessment of estimation errors). Finally, the model was used to generate a multiple input multiple output control scheme. As result, it was observed that the model had a correct representation of the phenomena involved and that the peripheral velocity and pressure used in the HPGR are useful manipulated variables to control the energy consumed by the equipment in an MPC scheme.