Model Validation Problem Research Articles

Properties of Optimal Solutions in L1 Identification Problem

This paper presents primal-dual formulation of a convex optimization problem related to l1 interpolatory-projection algorithm. The use of duality theory permits to obtain information on the structure of optimal solutions for l1 identi£cation problem and gives extra analytical tools for the analysis of identi£cation error and the estimation of a lowerbound for noise amplitude. The proposed analysis applies also to the case of l1 model validation problem and characterizes properties of uncertainty block and disturbance signal. A simulation example illustrates the results.

Stochastic Modelling of Random Variables with an Application in Financial Risk Management

The problem of determining whether or not a theoretical model is an accurate representation of an empirically observed phenomenon is one of the most challenging in the empirical scientific investigation. The following study explores the problem of stochastic model validation. Special attention is devoted to the unusual two-peaked shape of the empirically observed distributions of the conditional on realised volatility financial returns. The application of statistical hypothesis testing and simulation techniques leads to the conclusion that the conditional on realised volatility returns are distributed with a specific previously undocumented distribution. The probability density that represents this distribution is derived, characterised and applied for validation of the financial model.

Squared and absolute errors in optimal approximation of nonlinear systems

Optimal squared error and absolute error-based approximation problems for static polynomial models of nonlinear, discrete-time, systems are studied in detail. These problems have many similarities with other linear-in-the-parameters approximation problems, such as with optimal approximation problems for linear time-invariant models of linear and nonlinear systems. Nonprobabilistic signal analysis is used. Close connections between the studied approximation problems and certain classical topics in approximation theory, such as optimal L 2(−1,1) and L 1(−1,1) approximation, are established by analysing conditions under which sample averages of static nonlinear functions of the input converge to appropriate Riemann integrals of the static functions. These results should play a significant role in the analysis of corresponding system identification and model validation problems. Furthermore, these results demonstrate that optimal modelling based on the absolute error can offer advantages over squared error-based modelling. Especially, modelling problems in which some signals possess heavy tails can benefit from absolute value-based signal and error analysis.

An approach to identification for robust control

Given measured data generated by a discrete-time linear system, we propose a model consisting of a linear time-invariant system affected by norm-bounded perturbation. Under mild assumptions, the plants belonging to the resulting uncertain family form a convex set. The approach depends on two key parameters: an a priori given bound of the perturbation and the input used to generate the data. It turns out that the size of the uncertain family can be reduced by intersecting the model families obtained by making use of different inputs. The model validation problem in this identification scheme is analyzed. For a given energy level, the invalidation problem yields the family of those models which can never be invalidated for any possible input of fixed energy and any possible perturbation; this leads to the intersection of all uncertain families. A consequence of the invalidation problem is that for finite length measurements not all models can be invalidated, using fixed-energy inputs.

Aeroservoelastic Model Uncertainty Bound Estimation from Flight Data

Uncertainty modeling is a critical element in the estimation of robust stability margins for stability boundary prediction and robust flight control system development. There has been a serious deficiency to date in aeroservoelastic data analysis with attention to uncertainty modeling. Uncertainty can be estimated from flight data using both parametric and nonparametric identification techniques. The model validation problem addressed here is to identify aeroservoelastic models with associated uncertainty structures from a limited amount of controlled excitation inputs over an extensive flight envelope. The challenge is to update analytical models from flight data estimates while also deriving nonconservative uncertainty descriptions consistent with the flight data. Transfer function estimates are incorporated in a robust minimax estimation scheme to update models and get error bounds consistent with the data and model structure. Uncertainty estimates derived from the data in this manner provide an appropriate and relevant representation for model development and robust stability analysis. The method incorporates parametric and nonparamteric uncertainty into various uncertainty structures for quantitative measures of robust stability relating to parameter variations and unmodeled dynamics. This model-plus-uncertainty identification procedure is applied to aeroservoelastic flight data from the NASA Dryden Flight Research Center F-18 Systems Research Aircraft.

A nonlinear functional approach to LFT model validation

Model validation provides a useful means of assessing the ability of a model to account for a specific experimental observation, and has application to modeling, identification and fault detection. In this paper, we consider a new approach to the model validation problem by deploying quadratic functionals, and more generally nonlinear functionals, to specify noise and dynamical perturbation sets. Specifically, we consider a general linear fractional transformation framework for the model structure, and use constraints involving nonlinear functional inequalities to specify model non-linearities and unknown perturbations, and characteristics of noise and disturbance signals. Sufficient conditions for invalidation of such models are provided in terms of semidefinite programming problems.

INDUCED ONE-PARAMETER BIFURCATIONS IN IDENTIFIED NONLINEAR DYNAMICAL MODELS

It is shown that nonlinear global models identified from a single time series can be used to reproduce the same sequence of bifurcations of the original system. This has been observed for simulated and real data and for both difference equation and differential equation models, thus suggesting some generality. The results reported in this paper are of a practical character and seem to have some bearing not only on the important subject of estimating bifurcation diagrams from data, but also in model validation problems since some models can reproduce the bifurcation sequence of a system even when such models do not settle to the original attractor at first. In this case, models which would be dismissed are shown to display consistent dynamic information about the original system, as illustrated by a simulated and a real data example. An additional example that uses real data is provided in which the original bifurcation sequence is recovered by the addition of multiplicative noise with increasing variance.

A New Nonparametric Pairwise Clustering Algorithm Based on Iterative Estimation of Distance Profiles

We present a novel pairwise clustering method. Given a proximity matrix of pairwise relations (i.e. pairwise similarity or dissimilarity estimates) between data points, our algorithm extracts the two most prominent clusters in the data set. The algorithm, which is completely nonparametric, iteratively employs a two-step transformation on the proximity matrix. The first step of the transformation represents each point by its relation to all other data points, and the second step re-estimates the pairwise distances using a statistically motivated proximity measure on these representations. Using this transformation, the algorithm iteratively partitions the data points, until it finally converges to two clusters. Although the algorithm is simple and intuitive, it generates a complex dynamics of the proximity matrices. Based on this bipartition procedure we devise a hierarchical clustering algorithm, which employs the basic bipartition algorithm in a straightforward divisive manner. The hierarchical clustering algorithm copes with the model validation problem using a general cross-validation approach, which may be combined with various hierarchical clustering methods. We further present an experimental study of this algorithm. We examine some of the algorithm's properties and performance on some synthetic and ‘standard’ data sets. The experiments demonstrate the robustness of the algorithm and indicate that it generates a good clustering partition even when the data is noisy or corrupted.

NONLINEAR FUNCTIONAL CHARACTERIZATIONS OF UNCERTAINTY IN MODEL VALIDATION

Model validation provides a useful means of assessing the ability of a model to account for a specific experimental observation, and has application to modeling, identification and fault detection. In this paper we consider a new approach to the linear fractional transformation (LFT) model validation problem by deploying quadratic functionals, and more generally nonlinear functionals, to specify noise and dynamical perturbation sets. Sufficient conditions for invalidation of such models are provided in terms of semidefinite programming problems.

MODEL VALIDATION FOR IQC UNCERTAIN SYSTEMS WITH FIXED INITIAL CONDITIONS

The paper considers a model validation problem for a class of uncertain systems in which the uncertainty is described by an integral quadratic constraint and the uncertain system has zero initial condition. This leads to a method for model validation which is based around a robust Kalman filter type structure.

Model validation of multirate systems from time-domain experimental data

The model validation problem using time-domain experimental data is studied for multirate linear fractional uncertain models in this note. As a technical tool, the Caratheodory-Fejer (CF) interpolation problem with a nest operator constraint is first investigated. This problem is itself of interest mathematically and has potential applications in addressing other problems in control, signal processing, and circuit theory. A necessary and sufficient solvability condition for this interpolation problem is given. The validation tests are then presented based on this condition and the lifting technique. Tractable convex optimization methods can be used to solve the validation problems.

Fitting nonlinear low-order models for combustion instability control

Abstract Tools for fitting low-complexity nonlinear models based on experimental data are examined through the example problem of finding a reduced-order model suitable for control of a combustion instability operating in a limit cycle. This proceeds in four parts; physical modeling, linear system identification, nonlinear analysis, and validation test design. It is shown how the nonlinear tools of describing functions, bifurcation methods and manifold analysis assist in developing a simple nonlinear model capable of describing the data and consistent with physical understanding. The system being modeled is a lean gas turbine combustor which exhibits a sustained mid-range (100– 1000 Hz ) limit cycle instability. The closed-loop experimental data does not contain a sufficiently rich spectrum for confident modeling in the first linear system identification phase. Despite the paucity of information quality, a grey-box nonlinear model is created and parametrized which provides an explanation both of the limit-cycle fundamental oscillation and of a high frequency nonharmonic signal also present. The model structure is explored and various operating conditions simulated to understand the model better. The validation and/or refinement of this model is then considered. The model validation problem is important because of the poor information content of the periodic limit cycle data. The challenge is to provide a practically feasible, small excitation to the loop to improve identifiability and to provide qualitative tests of model performance. We examine this problem by considering the nonlinear dynamics of the model class and feasible excitation mechanisms.

Statistical analysis of a signal separation method based on second-order statistics

This correspondence explores a method for separation of dynamically mixed sources, which is based on second-order statistics. Here, a statistical analysis is given of a generalized version of the original algorithm. The generalized method includes a weighting matrix, and a result of the statistical analysis is that the best possible weighting is found. In cases where the sources have similar color, the weighted algorithm significantly improves the estimates of the mixing parameters. The problem of model validation is discussed as well.

Residual models and stochastic realization in state-space identification

This paper presents theory and algorithms for validation in system identification of state-space models from finite input-output sequences in a subspace model identification framework. Our formulation includes the problem of rank-deficient residual covariance matrices, a case which is encountered in applications with mixed stochastic-deterministic input-output properties as well as for cases where outputs are linearly dependent. Similar to the case of prediction-error identification, it is shown that the resulting model can be decomposed into an input-output model and a stochastic innovations model. Using the Riccati equation, we have designed a procedure to provide a reduced-order stochastic model that is minimal with respect to system order as well as the number of stochastic inputs thereby avoiding several problems appearing in standard application of stochastic realization to the model validation problem.

Closed-loop linear model validation and order estimation using polyspectral analysis

Suppose that we perform closed-loop linear system identification using polyspectral analysis given noisy time-domain input-output measurements. In this setup, it is assumed that various disturbances affecting the system are zero-mean stationary Gaussian, whereas the closed-loop system operates under an external (possibly noisy) non-Gaussian input. The closed-loop system must be stable, but it is allowed to be unstable in the open loop. Various techniques have been proposed for system identification using polyspectral analysis. Having obtained a model, how do we know if the fitted model is good? This paper is devoted to the problem of statistical model validation using polyspectral analysis. We propose simple statistical tests based on the estimated polyspectrum (integrated bispectrum and/or integrated trispectrum) of an output error signal or the estimated cross-polyspectrum between the external reference and the output error signal. Model order estimation is performed by repeatedly using the model validation procedure. Computer simulation examples are presented in support of the proposed approaches.

Approaches in identification and model validation for robust control design

Approaches of identification and model validation problem are investigated and for some model structures it is shown how the uncertainty overbounds can be calculated via some convex optimization problems in the form of LMFs. A certain sets of output noise and classes of model structures are considered and their properties are also transformed into LMI conditions. We have also proposed an iterative scheme that uses these approaches for robust control design. Simulation examples are given to show the implementability of these approaches.

LFT uncertain model validation with time- and frequency-domain measurements

In this paper the authors study a model validation problem pertaining to linear fractional uncertainty models. They extend the previous validation approaches, based upon either time or frequency measurements, to one using simultaneously time- and frequency-domain data. They show that this problem can be reduced to two independent convex feasibility tests, each of which corresponds to the time- or frequency-domain data alone, and each can be verified numerically using available algorithms and software programs. The merit of such a mixed time- and frequency-domain approach, therefore, lies in that not only can it accommodate the two distinctively different types of measurement data simultaneously, but also from a computational standpoint it will not suffer from a complexity growth beyond that associated with the existing time-domain tests.

Modelling drainage performance in slums of developing countries: how good is good enough?

A model for the dual drainage system in a flat 20 hectare slum in Indore, India was developed to evaluate the factors which influence drainage performance. Performance was defined by the depth, extent, duration, and frequency of flooding during the 1994 monsoon. This paper reports on the conceptual problem of model validation, particularly for “flooding” or “not flooding”. The model's predictions for maximum depth during verification were reasonable, as 72% of maximum depth predictions were within +/− 0.10 m of observed values. The predictions for the binary variable of flooding at a point were, however, poor. The model gave good predictions of freeboard violation, where the freeboard level was set at 50 mm below the lowest kerb. The paper offers some intuitive reasons as to why prediction of freeboard violation may be expected to be better than prediction of flooding. The authors conclude that where models cannot be validated for flooding prediction, it may be better to redefine performance criteria and validate the model for freeboard violation.

Model validation and order selection for linear model fitting using thirdand fourth-order cumulants

Given a linear stationary non-Gaussian signal, suppose that we fit a linear model using higher order statistics and one of several existing methods. The model is fitted under certain assumptions on the data and the underlying (true) model. Having obtained a model, how do we know if the fitted model is "good"? This paper is devoted to the problem of model diagnostics and validation. We propose frequency-domain tests that are applicable to both third- and fourth-order statistics based model fitting unlike existing tests. Model order selection is a byproduct of the model validation approach. Two computer simulation examples are presented to illustrate the proposed tests.

Model validation and state estimation for uncertain continuous-time systems with missing discrete-continuous data

This paper considers two related problems of state estimation and model validation for a class of uncertain linear systems. The main contribution of the paper is that it considers a general information structure which allows for discrete and continuous measurements as well as missing data. The results are given in terms of a recursive state estimator involving a jump Riccati differential equation and jump state equations. These equations can be solved on-line.