Model Structural Errors Research Articles

A probabilistic methodology for quantifying, diagnosing and reducing model structural and predictive errors in short term water demand forecasting

Accurate forecasts of water demand are required for real-time control of water supply systems under normal and abnormal conditions. A methodology is presented for quantifying, diagnosing and reducing model structural and predictive errors for the development of short term water demand forecasting models. The methodology (re-)emphasises the importance of posterior predictive checks of modelling assumptions in model development, and to account for inherent demand uncertainty, quantifies model performance probabilistically through evaluation of the sharpness and reliability of model predictive distributions. The methodology, when applied to forecast demand for three District Meter Areas in the UK, revealed the inappropriateness of simplistic Gaussian residual assumptions in demand forecasting. An iteratively revised, parsimonious model using a formal Bayesian likelihood function that accounts for kurtosis and heteroscedasticity in the residuals led to sharper yet reliable predictive distributions that better quantifies the time varying nature of demand uncertainty across the day in water supply systems.

Identification of Nonlinear Soft Sensor Models of Industrial Distillation Process under Uncertainty

The advancement of method of nonlinear soft sensor development using a nonparametric approach based on the example of industrial multicomponent distillation column is considered. The proposed procedure for soft-sensors evaluation is directed to overcome such difficulties in practice as process nonlinearity and uncertainty due to unknown model structure and measurement errors of output.

Estimation of Nonparametric Multiple Regression Measurement Error Models with Validation Data

In this article, we develop estimation approaches for nonparametric multiple regression measurement error models when both independent validation data on covariables and primary data on the response variable and surrogate covariables are available. An estimator which integrates Fourier series estimation and truncated series approximation methods is derived without any error model structure assumption between the true covariables and surrogate variables. Most importantly, our proposed methodology can be readily extended to the case that only some of covariates are measured with errors with the assistance of validation data. Under mild conditions, we derive the convergence rates of the proposed estimators. The finite-sample properties of the estimators are investigated through simulation studies.

A pathway analysis of global aerosol processes

Abstract. We present a detailed budget of the changes in atmospheric aerosol mass and numbers due to various processes: emission (including instant condensation of soluble biogenic emissions), nucleation, coagulation, H2SO4 condensation and in-cloud production, aging and deposition. The budget is created from monthly averaged tracer tendencies calculated by the global aerosol model ECHAM5.5-HAM2 and allows us to investigate process contributions at various length-scales and timescales. As a result, we show in unprecedented detail what processes drive the evolution of aerosol. In particular, we show that the processes that affect aerosol masses are quite different from those that affect aerosol numbers. Condensation of H2SO4 gas onto pre-existing particles is an important process, dominating the growth of small particles in the nucleation mode to the Aitken mode and the aging of hydrophobic matter. Together with in-cloud production of H2SO4, it significantly contributes to (and often dominates) the mass burden (and hence composition) of the hydrophilic Aitken and accumulation mode particles. Particle growth itself is the leading source of number densities in the hydrophilic Aitken and accumulation modes, with their hydrophobic counterparts contributing (even locally) relatively little. As expected, the coarse mode is dominated by primary emissions and mostly decoupled from the smaller modes. Our analysis also suggests that coagulation serves mainly as a loss process for number densities and that, relative to other processes, it is a rather unimportant contributor to composition changes of aerosol. The analysis is extended with sensitivity studies where the impact of a lower model resolution or pre-industrial emissions is shown to be small. We discuss the use of the current budget for model simplification, prioritization of model improvements, identification of potential structural model errors and model evaluation against observations.

Sensitivity of fluvial sediment source apportionment to mixing model assumptions: A Bayesian model comparison.

Mixing models have become increasingly common tools for apportioning fluvial sediment load to various sediment sources across catchments using a wide variety of Bayesian and frequentist modeling approaches. In this study, we demonstrate how different model setups can impact upon resulting source apportionment estimates in a Bayesian framework via a one-factor-at-a-time (OFAT) sensitivity analysis. We formulate 13 versions of a mixing model, each with different error assumptions and model structural choices, and apply them to sediment geochemistry data from the River Blackwater, Norfolk, UK, to apportion suspended particulate matter (SPM) contributions from three sources (arable topsoils, road verges, and subsurface material) under base flow conditions between August 2012 and August 2013. Whilst all 13 models estimate subsurface sources to be the largest contributor of SPM (median ∼76%), comparison of apportionment estimates reveal varying degrees of sensitivity to changing priors, inclusion of covariance terms, incorporation of time-variant distributions, and methods of proportion characterization. We also demonstrate differences in apportionment results between a full and an empirical Bayesian setup, and between a Bayesian and a frequentist optimization approach. This OFAT sensitivity analysis reveals that mixing model structural choices and error assumptions can significantly impact upon sediment source apportionment results, with estimated median contributions in this study varying by up to 21% between model versions. Users of mixing models are therefore strongly advised to carefully consider and justify their choice of model structure prior to conducting sediment source apportionment investigations.Key PointsAn OFAT sensitivity analysis of sediment fingerprinting mixing models is conductedBayesian models display high sensitivity to error assumptions and structural choicesSource apportionment results differ between Bayesian and frequentist approaches

A fractional order controller for seismic mitigation of structures equipped with viscoelastic mass dampers

In this paper a fractional order (FO) controller is proposed for solving the vibration suppression problem in civil structures. A laboratory scaled steel structure, with one floor, modeled as a single degree-of-freedom system is used as a case study. Two passive control solutions are proposed: a tuned mass damper (TMD) and a viscoelastic damper (VED), the latter being modeled using fractional derivatives. The simulation results show that the VED is able to further reduce the vibrations induced as forced oscillations or due to seismic excitation inputs, as compared to the passive TMD. The FO controller is then tuned using a new approach based on imposing a magnitude condition for the closed-loop system at the structural resonance frequency. The resulting FO active control strategy, together with the VED, ensures an increased seismic mitigation. Structural modeling errors are also considered, with the proposed active FO control strategy behaving robustly in terms of vibration suppression. The novelty of the paper resides in the tuning approach, as well as in the proposed active control strategy that is based upon combing VEDs, described using an FO model, and an FO controller.

Multi-model ensemble hydrologic prediction and uncertainties analysis

Abstract. Modelling uncertainties (i.e. input errors, parameter uncertainties and model structural errors) inevitably exist in hydrological prediction. A lot of recent attention has focused on these, of which input error modelling, parameter optimization and multi-model ensemble strategies are the three most popular methods to demonstrate the impacts of modelling uncertainties. In this paper the Xinanjiang model, the Hybrid rainfall–runoff model and the HYMOD model were applied to the Mishui Basin, south China, for daily streamflow ensemble simulation and uncertainty analysis. The three models were first calibrated by two parameter optimization algorithms, namely, the Shuffled Complex Evolution method (SCE-UA) and the Shuffled Complex Evolution Metropolis method (SCEM-UA); next, the input uncertainty was accounted for by introducing a normally-distributed error multiplier; then, the simulation sets calculated from the three models were combined by Bayesian model averaging (BMA). The results show that both these parameter optimization algorithms generate good streamflow simulations; specifically the SCEM-UA can imply parameter uncertainty and give the posterior distribution of the parameters. Considering the precipitation input uncertainty, the streamflow simulation precision does not improve very much. While the BMA combination not only improves the streamflow prediction precision, it also gives quantitative uncertainty bounds for the simulation sets. The SCEM-UA calculated prediction interval is better than the SCE-UA calculated one. These results suggest that considering the model parameters' uncertainties and doing multi-model ensemble simulations are very practical for streamflow prediction and flood forecasting, from which more precision prediction and more reliable uncertainty bounds can be generated.

Extension of the tuning constant in the Huber's function for robust modeling of piezoelectric systems

SummaryThis paper proposes a new modeling approach that is experimentally validated on piezoelectric systems in order to provide a black‐box pseudolinear model for complex systems control. Most of the time, one uses physical based approaches. However, sometimes complex phenomena occur in the system due to atypical changes of the process behavior, output noise or some hard nonlinearities. Therefore, we adopt identification methods to achieve the modeling task. The microdisplacements of the piezoelectric systems generate atypical data named outliers, leading to large estimated prediction errors. Since these errors disturb the classical normal probability density function, we choose here, as corrupted distribution model, the gross error model (GEM). In order to deal more efficiently with the outliers, we use the Huber's function, as mixed L2/L1 norms in which the tuning threshold named scaling factor is extended. From this function, a cost function also named PREC as parameterized robust estimation criterion is established. The identification is performed by choosing an Output Error model structure. In order to express the asymptotic covariance matrix of the robust estimator, we present a L finite Taylor's expansion to linearize the gradient and the hessian of the PREC. Experimental results are presented and discussed. Copyright © 2014 John Wiley & Sons, Ltd.

Kinematic Calibration of Hybrid Machine Tool for Marine Propellers Processing

To increase the precision and the processing efficiency of large-scale marine propeller, the method of twin-tool symmetrical processing is proposed based on hybrid mechanism. With the introduction of mechanism compositions of the machine tool, the calibration method of the structural parameters is researched. The serial and parallel mechanisms of the hybrid mechanism are calibrated separately for the coupled motion. Their structural error models are established and identification methods of structural parameters are analyzed. And nonlinear equations are solved by using the Levenberg-Marquardt optimization algorithm to obtain the structural parameters. Finally, in ADAMS, hybrid machine tool model with structural parameter errors is established. And the pose errors of the tool are acquired in kinematics simulation. After that, the structural parameters errors of the machine tool are compensated by calibrating. Then, the error models and the parameter identification methods are proved to be accurate. Meanwhile, the correctness and effectiveness of the calibration method is proved.

Inference of reactive transport model parameters using a Bayesian multivariate approach

AbstractParameter estimation of subsurface transport models from multispecies data requires the definition of an objective function that includes different types of measurements. Common approaches are weighted least squares (WLS), where weights are specified a priori for each measurement, and weighted least squares with weight estimation (WLS(we)) where weights are estimated from the data together with the parameters. In this study, we formulate the parameter estimation task as a multivariate Bayesian inference problem. The WLS and WLS(we) methods are special cases in this framework, corresponding to specific prior assumptions about the residual covariance matrix. The Bayesian perspective allows for generalizations to cases where residual correlation is important and for efficient inference by analytically integrating out the variances (weights) and selected covariances from the joint posterior. Specifically, the WLS and WLS(we) methods are compared to a multivariate (MV) approach that accounts for specific residual correlations without the need for explicit estimation of the error parameters. When applied to inference of reactive transport model parameters from column‐scale data on dissolved species concentrations, the following results were obtained: (1) accounting for residual correlation between species provides more accurate parameter estimation for high residual correlation levels whereas its influence for predictive uncertainty is negligible, (2) integrating out the (co)variances leads to an efficient estimation of the full joint posterior with a reduced computational effort compared to the WLS(we) method, and (3) in the presence of model structural errors, none of the methods is able to identify the correct parameter values.

A compatible system of biomass equations for three conifer species in Northeast, China

A compatible system of biomass equations was developed for three major conifer species, Korean spruce (Picea koraiensis Nakai), Korean pine (Pinus koraiensis Sieb. et Zucc), and Dahurian larch (Larix gmelinii Rupr) in northeastern China. The model error structure (additive vs multiplicative) of the power function (Y=a Xb) was evaluated using a likelihood analysis. The results indicated that the assumption of multiplicative error structure was strongly supported for the biomass equations of total, sub-total, and tree components. Thus, a system of log-transformed biomass equations was developed using nonlinear seemly unrelated regression (NSUR), with three constraints on the structural parameters to account for the cross-equation error correlations between four tree component biomass (roots, stems, branches, and foliage), two sub-total biomass (aboveground and crown), and total tree biomass. The effectiveness of three anti-log correction factors for predicting the expected biomass in original scale was also assessed. Our results indicated that (1) the likelihood analysis can be used as a tool for rigorously evaluating the error structures of tree biomass equations and choosing an appropriate model form for the given biomass data; (2) the additive or compatible system of biomass equations with three constraints can be developed by NSUR to obtain favorable model fitting and prediction performance; and (3) the anti-log correction may not be necessary and sometimes can be ignored.

Uncertainty and sensitivity analyses using GLUE when modeling inhibition and pharmaceutical cometabolism during nitrification

The influence of uncertainties in biokinetic parameters for ammonia and nitrite oxidizing bacteria on the performance of a two-step nitrification model was evaluated using the Generalized Uncertainty Estimation (GLUE) technique. The predictive capability of behavioral simulations generated using GLUE was assessed utilizing data from experiments comparing nitrification in the presence and absence of two pharmaceuticals – atenolol or sotalol. Results suggest that GLUE cannot account for model structural error arising when ammonia oxidation is competitively inhibited. Use of a competitive inhibition model for ammonia oxidation (i.e., correction of the model structural error), however, enables GLUE to generate meaningful uncertainty intervals. While GLUE is used in the present study, other uncertainty analysis techniques are likely to be similarly unable to account for model structural errors. Thus, results from this study emphasize the importance of model selection for efficacious uncertainty analysis. The behavioral simulations generated using GLUE based on application of the correct model was subsequently used to evaluate the sensitivity of transformation coefficients employed to describe the cometabolism of atenolol by ammonia oxidizing bacteria (AOB). Sensitivity was assessed by computing nonparametric elasticities of the cometabolism transformation coefficients to biokinetic parameters selected to describe nitrification in a novel application of a generalized nonparametric analysis. Results suggest that the AOB-growth related transformation coefficient of atenolol is relatively insensitive to variation in ammonia and nitrite oxidizing biokinetic parameters. In contrast, the non-growth related transformation coefficient describing atenolol cometabolism appears to be sensitive to the specific growth rate of AOB. Elasticities are used to assess whether estimates of atenolol-AOB cometabolic biodegradation coefficients from lab-scale experiments could be used more generally. This novel application of elasticities to biological wastewater process modeling suggests that seasonal temperature variations may be an important factor in pharmaceutical biodegradation during biological wastewater treatment.

Estimating the Uncertainty of Hydrological Predictions through Data-Driven Resampling Techniques

AbstractEstimating the uncertainty of hydrological models remains a relevant challenge in applied hydrology, mostly because it is not easy to parameterize the complex structure of hydrological model errors. A nonparametric technique is proposed as an alternative to parametric error models to estimate the uncertainty of hydrological predictions. Within this approach, the above uncertainty is assumed to depend on input data uncertainty, parameter uncertainty and model error, where the latter aggregates all sources of uncertainty that are not considered explicitly. Errors of hydrological models are simulated by resampling from their past realizations using a nearest neighbor approach, therefore avoiding a formal description of their statistical properties. The approach is tested using synthetic data which refer to the case study located in Italy. The results are compared with those obtained using a formal statistical technique (meta-Gaussian approach) from the same case study. Our findings prove that the nea...

Data Worth of Measurement Quantities and Uncertainty Considerations in Lithium-Ion Battery Modeling

Because of their favorable properties like, e.g., high energy density, lithium-ion batteries are ubiquitous in modern societies. Still, some security aspects, like thermal runaway, are not yet fully solved [1]. As a first step towards an early-alert system that can warn the user of an imminent thermal runaway we investigate some of the stochastic aspects of thermal battery model predictions.Numerical system modeling is inherently challenged by system uncertainties. These uncertainties stem from incomplete knowledge about parameter values, error-prone measurements and the imperfect mathematical description of the real system. The latter results in differences between predicted model output and the behavior of the real system. This phenomenon is called model structural error.We present a minimally intrusive methodology that allows to handle model errors of an existing thermal-electrochemical battery model in a stochastically rigorous way. Moreover, we investigate the explanatory power of different measurement quantities on the temperature profile inside of the battery.An existing, thermal-electrochemical battery model (DENIS [2]) of a LiFePO4 cell is extended by an additional, stochastically motivated model for the model structural error. The parameters for this error model are determined by means of a Monte Carlo maximum likelihood approach. That is, the residuals between experimental discharge curves at low temperatures and the model predictions are used to evaluate the likelihood of different error model parameter sets.Based on this stochastic system description a particle filter is implemented. Instead of a single model run, several parallel runs are computed. This yields a probability density function of the temperature distribution inside of the battery. Online measurement data can be used to update the system state and to decrease the uncertainty of future predictions. In comparison to Kalman filters, the use of a particle filter allows for a wide choice of error models beyond Gaussian and linear system assumptions.The development of this methodology is extended by a data worth analysis. By application of the PreDIA (“Preposterior Data Impact Assessor”) method [3] we investigate the explanatory power of different measurement quantities on the temperature profile inside of the battery. Voltage, discharge current and temperature measurements are considered and for the latter also different sensor locations are examined. The dynamics of the stochastic system description and the relative data worth of the different measurement quantities are interpreted and an outlook towards an early-alert system that builds upon the presented methodology is presented.[1] Bandhauer, Todd M., Srinivas Garimella, and Thomas F. Fuller. "A critical review of thermal issues in lithium-ion batteries." Journal of the Electrochemical Society 158.3 (2011): R1-R25.[2] Neidhardt, Jonathan P., et al. "A Flexible Framework for Modeling Multiple Solid, Liquid and Gaseous Phases in Batteries and Fuel Cells." Journal of The Electrochemical Society 159.9 (2012): A1528-A1542.[3] Leube, P. C., A. Geiges, and W. Nowak. "Bayesian assessment of the expected data impact on prediction confidence in optimal sampling design." Water Resources Research 48.2 (2012).

The Potential for Self-Organizing Maps to Identify Model Error Structures

Abstract An important aspect of numerical weather model improvement is the identification of deficient areas of the model, particularly deficiencies that are flow dependent or otherwise vary in time or space. Here the authors introduce the use of self-organizing maps (SOMs) and analysis increments from data assimilation to identify model deficiencies. Systematic increments reveal time- and space-dependent systematic errors, while SOMs provide a method for categorizing forecasts or increment patterns. The SOMs can be either used for direct analysis or used to produce composites of other fields. This study uses the forecasts and increments of 2-m temperature and dry column mass perturbation μ over a 4-week period to demonstrate the potential of this technique. Results demonstrate the potential of this technique for identifying spatially varying systematic model errors.

Parametric sensitivity and calibration for the Kain‑Fritsch convective parameterization scheme in the WRF model

Convective parameterizations used in weather and climate models all display sensitivity to model resolution and variable skill in different climatic regimes. Although parameters in convective schemes can be calibrated using observations to reduce model errors, it is not clear if the optimal parameters calibrated based on regional data can robustly improve model skill across different model resolutions and climatic regimes. In this study, this issue is investigated using a regional modeling framework based on the Weather Research and Forecasting (WRF) model. To quantify the response and sensitivity of model performance to model parameters, we identified five key input parameters and specified their ranges in the Kain-Fritsch (KF) convection scheme in WRF and calibrated them across different spatial resolutions, climatic regimes, and radiation schemes using observed precipitation data. Results show that the optimal values for the five input parameters in the KF scheme are close and model sensitivity and error exhibit similar dependence on the input parameters for all experiments conducted in this study despite differences in the precipitation climatology. We found that the model overall performances in simulating precipitation are more sensitive to the coefficients of downdraft (Pd) and entrainment (Pe) mass flux and starting height of downdraft (Ph). However,more » we found that rainfall biases, which are probably more related to structural errors, still exist over some regions in the simulation even with the optimal parameters, suggesting further studies are needed to identify the sources of uncertainties and reduce the model biases or structural errors associated with missed or misrepresented physical processes and/or potential problems with the modeling framework.« less

Evaluation of multi-model simulated soil moisture in NLDAS-2

Summary The North American Land Data Assimilation System (NLDAS) phase 2 (NLDAS-2) has generated 31-years (1979–2008) of water and energy products from four state-of-the-art land surface models (Noah, Mosaic, SAC, VIC). The soil moisture data from these models have been used for operational drought monitoring activities, but so far have not yet been comprehensively evaluated. In this study, three available in situ soil moisture observation data sets in the United States were used to evaluate the model-simulated soil moisture for different time scales varying from daily to annual. First, we used the observed multiple layer monthly and annual mean soil moisture from the Illinois Climate Network to evaluate 20-years (January 1985–December 2004) of model-simulated soil moisture in terms of skill and analysis of error statistics. Second, we utilized 6-years (1 January 1997–31 December 2002) of daily soil moisture observed from 72 sites over the Oklahoma Mesonet network to assess daily and monthly simulation skill and errors for 3 model soil layers (0–10 cm, 10–40 cm, 40–100 cm). Third, we extended the daily assessment to sites over the continental United States using 8-years (1 January 2002–31 December 2009) of observations for 121 sites from the Soil Climate Analysis Network (SCAN). Overall, all models are able to capture wet and dry events and show high skill (in most cases, anomaly correlation is larger than 0.7), but display large biases when compared to in situ observations. These errors may come from model errors (i.e., model structure error, model parameter error), forcing data errors, and in situ soil moisture measurement errors. For example, all models simulate less soil moisture due to lack of modeled irrigation and ground water processes in Illinois, Oklahoma, and the other Midwest states.

Optimization and robustness analysis of hybridoma cell fed-batch cultures using the overflow metabolism model

The maximization of biomass productivity in fed-batch cultures of hybridoma cells is analyzed based on the overflow metabolism model. Due to overflow metabolism, often attributed to limited oxygen capacity, lactate and ammonia are formed when the substrate concentrations (glucose and glutamine) are above a critical value, which results in a decrease in biomass productivity. Optimal feeding rate, on the one hand, for a single feed stream containing both glucose and glutamine and, on the other hand, for two separate feed streams of glucose and glutamine are determined using a Nelder-Mead simplex optimization algorithm. The optimal multi exponential feed rate trajectory improves the biomass productivity by 10 % as compared to the optimal single exponential feed rate. Moreover, this result is validated by the one obtained with the analytical approach in which glucose and glutamine are fed to the culture so as to control the hybridoma cells at the critical metabolic state, which allows maximizing the biomass productivity. The robustness analysis of optimal feeding profiles obtained with different optimization strategies is considered, first, with respect to parameter uncertainties and, finally, to model structure errors.

Pseudo-Orbit Data Assimilation. Part II: Assimilation with Imperfect Models

Abstract Data assimilation and state estimation for nonlinear models is a challenging task mathematically. Performing this task in real time, as in operational weather forecasting, is even more challenging as the models are imperfect: the mathematical system that generated the observations (if such a thing exists) is not a member of the available model class (i.e., the set of mathematical structures admitted as potential models). To the extent that traditional approaches address structural model error at all, most fail to produce consistent treatments. This results in questionable estimates both of the model state and of its uncertainty. A promising alternative approach is proposed to produce more consistent estimates of the model state and to estimate the (state dependent) model error simultaneously. This alternative consists of pseudo-orbit data assimilation with a stopping criterion. It is argued to be more efficient and more coherent than one alternative variational approach [a version of weak-constraint four-dimensional variational data assimilation (4DVAR)]. Results that demonstrate the pseudo-orbit data assimilation approach can also outperform an ensemble Kalman filter approach are presented. Both comparisons are made in the context of the 18-dimensional Lorenz96 flow and the two-dimensional Ikeda map. Many challenges remain outside the perfect model scenario, both in defining the goals of data assimilation and in achieving high-quality state estimation. The pseudo-orbit data assimilation approach provides a new tool for approaching this open problem.

Laplace’s Demon and the Adventures of His Apprentices

The sensitive dependence on initial conditions (SDIC) associated with nonlinear models imposes limitations on the models’ predictive power. We draw attention to an additional limitation than has been underappreciated, namely, structural model error (SME). A model has SME if the model dynamics differ from the dynamics in the target system. If a nonlinear model has only the slightest SME, then its ability to generate decision-relevant predictions is compromised. Given a perfect model, we can take the effects of SDIC into account by substituting probabilistic predictions for point predictions. This route is foreclosed in the case of SME, which puts us in a worse epistemic situation than SDIC.