Local Calibration Methods Research Articles

On the implementation of post-processing of runoff forecast ensembles

AbstractDifferent post-processing techniques are frequently employed to improve the outcome of ensemble forecasting models. The main reason is to compensate for biases caused by errors in model structure or initial conditions, and as a correction for under- or overdispersed ensembles. Here we use the Ensemble Model Output Statistics method to post-process the ensemble output from a continental scale hydrological model, LISFLOOD, as used in the European Flood Awareness System (EFAS). We develop a method for local calibration and interpolation of the post-processing parameters and compare it with a more traditional global calibration approach for 678 stations in Europe based on long term observations of runoff and meteorological variables. For the global calibration we also test a reduced model with only a variance inflation factor. Whereas the post-processing improved the results for the first 1-2 days lead time, the improvement was less for increasing lead times of the verification period. This was the case both for the local and global calibration methods. As the post-processing is based on assumptions about the distribution of forecast errors, we also present an analysis of the ensemble output that provides some indications of what to expect from the post-processing.

Integrated Highway Safety Information System (HSIS) datasets by means of a Roadway Safety Data Integrator (RSDI) tool.

The datasets and tool presented in this article are related to the research article entitled “Improving crash predictability of the Highway Safety Manual through optimizing local calibration process” (Dadvar et al., 2020) [1], in which these datasets were used to investigate alternative local calibration methods for the Highway Safety Manual (HSM) [2] local application. The datasets are integrated Highway Safety Information System (HSIS) [3] state-wide rural two-lane, two-way roads (R2U) data from states of Illinois and Washington.The HSIS is a database that maintains motor vehicle crash data, roadway inventory, and traffic volume data for several US states. It is an excellent source of data to highway safety research and can be used to investigate many research questions. However, to prepare an analysis-ready roadway safety dataset based on the HSIS or any databases that store the relevant data in multiple different datasets, the researchers should integrate multiple datasets, merge or unmerge and remove certain inconsistent records, and finally clean the integrated dataset. The HSIS staff is usually accommodating and eager to help, but sometimes the nature of data needs is complicated and laborious.A tool named Roadway Safety Data Integrator (RSDI) was developed for combining, segmenting, and selecting homogeneous (unchanged during the study period for certain variables of interest) HSIS roadway segments and also crash assignment by desired crash attributes (e.g., crash severity or type). The RSDI tool can be helpful for integrating different safety-related datasets such as roadway inventory (including grade, curve, and other subsets), traffic volume, and motor vehicle crash data; also, it can do required segmentation and identify the homogeneous roadway segments over the desired years of study that are the basis for development and calibration of the HSM predictive models.The shared datasets contain homogeneous roadway segments, geometric details, and crash data for six years from Illinois (2005–10) and Washington (2010–15). The datasets and RSDI tool would be important sources generally for investigating highway safety research questions and in particular, for HSM-related analyses. The RSDI tool can be used for similar purposes and it is not limited to the HSIS data. It can be used for segmentation and finding homogeneous segments of any datasets that follow linear referencing.The datasets and RSDI tool are hosted in the Mendeley Data repository [4].

Effects of various partitions on the accuracy of virtual in-situ calibration in building energy systems

Sensor errors significantly affect the operation and control of building energy systems. Thus, using correct and reliable sensors can effectively reduce the energy consumption of building energy systems. Virtual in-situ calibration (VIC), which is based on Bayesian inference and the Markov Chain Monte Carlo method, can help avoid the installation of new sensors, effectively reduce the systematic and random errors of sensors, and improve the reliability of collected data. In the current study, six working conditions were designed to check the robustness and accuracy of the proposed VIC technology. It's shown that for the systematic and random error of various sensors, the improved component calibration method is the most accurate. Contrarily, the whole calibration method is the least accurate, whereas the performance of the three local calibrations is intermediate. The maximum errors of the whole, local, and component calibration methods were 973%, 112.7%, and 30%, respectively. This verifies that component calibration effectively reduces the possibility of abnormal data and also enhances the reliability of sensor measurements. For the whole and local calibration methods, the constraints between various parameters are significantly reduced because of the random and uncertain coefficients; consequently, a majority of the correction results deviate from their true values. It is not necessary to determine the sensitivity and normalizing coefficients through a complicated optimization algorithm or historical experience by using the proposed component calibration. Therefore, the component calibration method features advantages in terms of computational time and correction accuracy.

Comparing local calibration using random effects estimation and Bayesian calibrations: a case study with a mixed effect stem profile model

Local site-level calibration of allometric models was scrutinized. Two Bayesian calibration methods were compared to local random effects estimations. The Bayesian calibration methods proved more effective than local estimation of random effects in reducing prediction bias. The simplest literature-based calibration can be recommended. The local calibration had minor effects on stem volume estimations. The spatial variability of trees allometry has long prompted the necessity for local, site- or stand-level calibrations. Mixed-effect models have enabled a quantitative progress through a local calibration where data are available. More recently, Bayesian statistics brought new alternatives owing to their formal definition of the random effects from prior information. To compare three local calibration methods: (i) a calibration based on the estimation of the local random effects, (ii) a Bayesian calibration where prevailing measurements are used to produce prior estimations, and (iii) a Bayesian calibration reproducing a calibration based on literature data only. The three calibrations were compared using a stem taper model developed for Norway spruce in Romania. The taper model was fitted to a large dataset, then applied locally to two high-elevation sites with contrasting growing conditions. The local calibration of mixed-effect models resulted in small gains and high biases. The Bayesian calibrations yielded better results, mostly because the Monte Carlo Markov Chain implementation permitted to tune of all the model’s parameters simultaneously. The differences in stem volume estimations were however always very small ranging from − 5.2 to 3.3% of the non-calibrated volume. The Bayesian literature-like calibration performed as well as the calibration using the large dataset (4–97% bias reduction according to the tree) and can be preferred for its ease of use.

Approaches for local calibration of mechanistic-empirical pavement design guide joint faulting model: a case study of Ontario

ABSTRACT The Mechanistic-Empirical Pavement Design Guide (MEPDG) has been employed by agencies as an innovative method for pavement design since the National Cooperative Highway Research Program (NCHRP) Project 1-37A was implemented in 2004. Over the years, the MEPDG has evolved into the AASHTOWare Pavement ME Design software (AASHTOWare®). Local calibration of the performance models in the AASHTOWare® is a crucial and challenging task to improve the effectiveness of its application. The accuracy of the calibration depends on efficient methods and validation processes. This paper aims at developing local calibration methods for joint faulting prediction model of Jointed Plain Concrete Pavement (JPCP). This study not only focuses on improving the prediction accuracy of the joint faulting model but also demonstrating the various optimisation procedures in detail. A total of 27 representative JPCP sections were used in the processes of calibration. Three optimisation approaches were used: (1) One-At-a-Time (OAT) through the trial-and-error procedure, (2) generalised reduced gradient (GRG) using MS Excel® Solver, and (3) Levenberg-Marquardt Algorithm (LMA) fitting the functions. The prediction accuracy of local models was improved as compared with the global ones. Average Bias (AB) reduced from 0.3083 to 0.0578, and Standard Error of the Estimate (SEE) reduced from 0.3345 to 0.1912. Among the three local calibration approaches, approach 2 and approach 3 had more significant improvement on results than approach 1. Finally, the integral procedures were provided for local calibration in Ontario.

Local calibration of creep compliance models of asphalt concrete

Thermal cracking is one of the main asphalt concrete distresses. Asphalt pavements typically experience thermal cracking at an early stage of their service life. The Mechanistic-Empirical Pavement Design (MEPDG) software utilizes a thermal cracking model to predict low temperature performance (NCHRP, 2004). Creep compliance of the asphalt concrete mixture is an essential input to the thermal cracking model. Creep input values are generated by a globally calibrated creep compliance model, based on certain binder and mixture properties. Local calibration is necessary to improve the model performance to account for local materials, environment and other factors.This paper presents the results of two alternative local calibration methods; nonlinear multiple regression and artificial neural network (ANN), and compares their performance to the MEPDG generalized model. In total, 861 data points from 41 specimens representing 14 types of AC mixes were prepared and tested in the laboratory. The creep compliance values were determined in the laboratory and compared to values generated by globally and locally calibrated models. The analysis shows that the MEPDG creep compliance model without local calibration predicts creep values poorly, thus calibration should be a required step in the implementation of MEPDG designs. As expected, both nonlinear regression and ANN model significantly improve the reliability of creep compliance predictions. The ANN model produced the lowest error in creep compliance predictions.

Do affine jump-diffusion models require global calibration? Empirical studies from option markets

This study presents an empirical evaluation of the efficiency and robustness of calibration methods for option pricing models with closed-form expression, specifically by using affine jump-diffusion models. To mitigate the local minima problems inherent in model calibration, we provide a global calibration method with enhanced discrete local search strategy. We compare this global calibration method with local calibration methods by using both model generated and real-market cross-sectional option data. Global calibration is highly essential for obtaining a reliable parameter set for affine jump-diffusion models and yields robust calibration results. By using the 2007 S&P 500 index options, we apply the global calibration method to 12 widely used affine jump-diffusion models. Results show that the calibrated parameters common across the models share similar values. This result verifies the robustness of global calibration. Conversely, local calibration generates different values. We also examine the predictive performance of the models and find that global calibration outperforms local calibration for the considered affine jump-diffusion models with respect to in-sample and out-of-sample errors.

Quantitative Measurements of Anaerobic Digestion Process Parameters Using near Infrared Spectroscopy and Local Calibration Models

The performance of local calibration models for quantitative measurements of ammonium and acetate on samples from an anaerobic digestion process was examined. The local calibration methods used were locally weighted regression (LWR) and multi-layer partial least squares (ML-PLS) regression. The results of these two methods were compared to each other and to the results from the global partial least squares (PLS) model regression as well. For ammonium, both the local methods performed excellently in comparison with global PLS models. However, the results from the 150 LWR models regressed for ammonium also showed that the accuracy can be highly dependent on the different combination alternatives for model parameter settings and pre-processing alternatives. For this reason, a number of distance measures were evaluated as local subset selection methods in ML-PLS. The benefits of an optimised layer structure and the iterative approach in ML-PLS were also evaluated for ammonium. This showed that some benefits can be obtained by optimising the layer structure, at least in the sense that the number of layers can be reduced, and that there can be a significant advantage in using an iterative approach in the selection of the local subset of calibration data. The local calibration methods were also evaluated for acetate but, in this case, the benefits compared to global PLS calibration models were fairly insignificant with ML-PLS and none at all with LWR.

Inverse, Classical, Empirical and Non-Parametric Calibrations in a Bayesian Framework

The calibration paradigm for near infrared (NIR) spectroscopy is explored in a Bayesian framework in which a model for the dependence of the NIR spectrum on the composition of the sample is combined with a prior distribution representing beliefs about the composition of the sample to be predicted. With appropriate choices for this prior distribution, it is possible to reproduce standard regression results, remove the shrinkage to the mean that is sometimes seen as a problem in inverse regression, or improve predictions by taking account of non-standard distributions of composition in the population of interest. These options are illustrated by applying them to the prediction of wheat and sunflower content using a database of 7532 commercial animal feed samples which has been studied extensively in the past. Various options for the modelling of the relationship between spectra and composition are investigated, including linear and non-linear regressions and a non-parametric approach based on kernels. This Bayesian kernel approach, which has features in common with local calibration methods, gave results better than anything previously achieved with the animal feed database.

Advanced Nonlinear Approaches for Predicting the Ingredient Composition in Compound Feedingstuffs by Near-Infrared Reflection Spectroscopy

For quantitative applications, the most common usage of near-infrared reflection spectroscopy (NIRS) technology, calibration involves establishing a mathematical relationship between spectral data and data provided by the reference. This model may be fairly complex, since the near-infrared spectrum is highly variable and contains physical/chemical information for the sample that may be redundant, and multivariate calibration is usually required. When the relationship to be modeled is nonlinear, classical regression methods are inadequate, and more complex strategies and algorithms must be sought in order to model this nonlinearity. The development of NIRS calibrations to predict the ingredient composition, i.e., the inclusion percentage of each ingredient, in compound feeds is a complex task, due to the nature of the parameters to be predicted and to the heterogeneous nature of the matrices/formulas in which each ingredient participates. The present paper evaluates the use of least squares support vector machines (LSSVM) and two local calibration methods, CARNAC and locally biased regression, for developing NIRS models to predict two of the most representative ingredients in compound feed formulations, wheat and sunflower meal, using a large spectral library of 7523 commercial compound feed samples. For both ingredients, the best results were obtained using CARNAC, with standard errors of prediction (SEP) of 1.7% and 0.60% for wheat and sunflower meal, respectively, and even better results when the algorithm was allowed to refuse to predict 10% of the unknowns. Meanwhile, LSSVM performed less well on wheat (SEP 2.6%) but comparably on sunflower meal (SEP 0.60%), giving results very similar to those reported previously for artificial neural networks. Locally biased regression was the least successful of the three methods, with SEPs of 3.3% for wheat and 0.72% for sunflower meal. All the nonlinear methods improved on the standard approach using partial least squares (PLS), which gave SEPs of 5.3% for wheat and 0.81% for sunflower meal.

Locally-Biased Regression

After a brief review of local calibration methods, a new and relatively simple method is proposed. Given a database of calibration samples, a global calibration based on all these samples and an unknown for which we wish to make a prediction, the method selects a subset of the calibration samples judged to be spectrally similar to the unknown and uses these to determine either a skew and bias or a simple bias correction to the global calibration. Spectral similarity is defined in a two-dimensional space, with one axis focussing on similarity with respect to the analyte value to be predicted, and the other on more general spectral similarity. The computations required to make a prediction are simple by the standards of local methods.

Calibration of reference materials for total-reflection X-ray fluorescence analysis by heavy ion backscattering spectrometry

Total-reflection X-ray fluorescence (TXRF) is widely used for the control of metallic contamination caused by surface preparation processes and silicon materials. At least three companies supply a variety of TXRF systems to the silicon integrated circuit (IC) community, and local calibration of these systems is required for their day to day operation. Differences in local calibration methods have become an issue in the exchange of information between IC manufacturers' different FABs (Fabrication Facility) and also between silicon suppliers and IC FABs. The question arises whether a universal set of fluorescence yield curves can be used by these different systems to scale system sensitivity from a single element calibration for calculation of elemental concentrations. This is emphasized by the variety of experimental conditions that are reported for TXRF data (e.g. different angles of incidence for the same X-ray source, different X-ray sources, etc.). It appears that an instrumental factor is required. We believe that heavy ion backscattering spectrometry (HIBS) provides a fundamental method of calibrating TXRF reference materials, and can be used in calculating this instrumental factor. In this paper we briefly describe the HIBS system at the Sandia National Laboratories HIBS User Facility and its application to the calibration of TXRF reference materials. We will compare HIBS and TXRF mapping capabilities and discuss the issues associated with the restrictions of some older TXRF sample stages. We will also discuss Motorola's cross-calibration of several TXRF systems using different elements as references.

The local calibration method for robot inaccuracy compensation

AbstractThis article presents a local calibration method that is philosophically different from the more contemporary global calibration methods. Local methods seek models of robot performance at the robot distal link measured relative to parts in localized part regions. In contrast, global methods seek to model various sources of robot inaccuracy internal to the robot. For example, a significant global research area is the determination of the actual robot arm structural parameters. Given more accurate estimates of these parameters, global compensation methods propose perturbation techniques or improved kinematic models that can be used to control the as‐manufactured robot. To implement local calibration methods, calibration procedures have been developed to locate tool and sensor tool control frames (TCF's) and to measure robot inaccuracy in localized regions. Using the measured position and orientation (pose) data obtained by hardware sensors, the components of operational inaccuracy are integrated into compensation models. This article discusses compensating model effectiveness and considers the interactions between hardware sensors and part features necessary to extract robot pose measurements automatically and effectively. Applying the relative calibration methodologies presented in this article to a GMF S‐200 six‐axis robot, robot repeatability was reduced by 39% and average localized robot inaccuracy was reduced from 3.4 mm (0.134 inch) to less than 0.86 mm (0.034 inch). The compensated robot inaccuracy is near the unstructured robot repeatability.