PREDICTION INTERVAL ESTIMATION TECHNIQUES FOR EMPIRICAL MODELING STRATEGIES AND THEIR APPLICATIONS TO SIGNAL VALIDATION TASKS

Abstract

The basis of this work was to evaluate both parametric and non-parametric empirical modeling strategies applied to signal validation or on-line monitoring tasks. On-line monitoring methods assess signal channel performance to aid in making instrument calibration decisions, enabling the use of condition-based calibration schedules. The three non-linear empirical modeling strategies studied were: artificial neural networks (ANN), neural network partial least squares (NNPLS), and local polynomial regression (LPR). The evaluation of the empirical modeling strategies includes the presentation and derivation of prediction intervals for each of three different model types studied. An estimate and its corresponding prediction interval contain the measurements with a specified certainty, usually 95%. The prediction interval estimates were compared to results obtained from bootstrapping via Monte Carlo resampling, to validate their expected accuracy. Properly determined prediction interval estimates were obtained that consistently captured the uncertainty of the given model such that the level of certainty of the intervals closely matched the observed level of coverage of the prediction intervals over the measured values. In most cases the expected level of coverage of the measured values within the prediction intervals was 95%. The prediction intervals were required to perform adequately under conditions of model misspecification. The results also indicate that instrument channel drifts are identifiable by observing the drop in the level of coverage of the prediction intervals to relatively low values, e.g. 30%. A comparative evaluation of the different empirical models was also performed. The evaluation considers the average estimation errors and the stability of the models under repeated Monte Carlo resampling. The results indicate the large uncertainty of ANN models applied to collinear data, and the utility of the NNPLS model for the same purpose. While the results from the LPR models remained consistent for data with or without collinearity, assuming proper regularization was applied. All of the methods studied herein were applied to a simulated data set for an initial evaluation of the methods, and data from two different U.S. nuclear power plants for the purposes of signal validation for on-line monitoring tasks.