Polynomial NARX Models Research Articles

Research on data‐driven model for soft sensing of natural gas production system

AbstractIn view of the problems of high cost and low reliability in obtaining operation information such as flow rate and pressure of offshore natural gas production system, research on soft sensing is carried out, and a dynamic data‐driven model bank is established, in purpose of estimating single‐well flow rate and wellhead pressure, providing convenience tool for online monitoring and system safety analysis. Combining dynamic and steady‐state samples, introducing black‐box identification techniques including orthogonal least square regression and deep learning along with parameter correction techniques such as bi‐objective least square algorithm and transfer learning, a series of nonlinear auto‐regressive models with exogenous inputs (NARX) are built, consisting of black‐box and gray‐box polynomial NARX (Poly‐NARX) models as well as deep neural network NARX (DNN‐NARX) models, approximately describing the dynamic performance of gas production well. Through realistic operation data, the simulation results of Poly‐NARX, DNN‐NARX, and multiple‐layer‐perception‐NARX models are compared. It is observed that gray‐box DNN‐NARX model shows the best performance with advantages of higher global applicability, better approximation ability, and stronger generalization ability. Proposed model bank is of high expansibility and engineering applicability for soft sensing problems in the petroleum industry, laying the ground work for building smart oil and gas field.

Applying polynomial decoupling methods to the polynomial NARX model

System identification uses measurements of a dynamic system’s input and output to reconstruct a mathematical model for that system. These can be mechanical, electrical, physiological, among others. Since most of the systems around us exhibit some form of nonlinear behavior, nonlinear system identification techniques are the tools that will help us gain a better understanding of our surroundings and potentially let us improve their performance. One model that is often used to represent nonlinear systems is the polynomial NARX model, an equation error model where the output is a polynomial function of the past inputs and outputs. That said, a major disadvantage with the polynomial NARX model is that the number of parameters increases rapidly with increasing polynomial order. Furthermore, the polynomial NARX model is a black-box model, and is therefore difficult to interpret. This paper discusses a decoupling algorithm for the polynomial NARX model that substitutes the multivariate polynomial with a transformation matrix followed by a bank of univariate polynomials. This decreases the number of model parameters significantly and also imposes structure on the black-box NARX model. Since a non-convex optimization is required for this identification technique, initialization is an important factor to consider. In this paper the decoupling algorithm is developed in conjunction with several different initialization techniques. The resulting algorithms are applied to two nonlinear benchmark problems: measurement data from the Silver-Box and simulation data from the Bouc-Wen friction model, and the performance is evaluated for different validation signals in both simulation and prediction.

Initialization Approach for Decoupling Polynomial NARX Models Using Tensor Decomposition

The Nonlinear Auto-regressive eXogenous input (NARX) model has been widely used in nonlinear system identification. It’s chief disadvantages are that it is a black-box model that suffers from the curse of dimensionality, in that the number of parameters increases rapidly with the nonlinearity degree. One approach to dealing with these problems involves decoupling the nonlinearity, but this requires solving a non-convex optimization problem. Solving non-convex optimization problems has always been challenging due to the possibility of getting trapped in a sub-optimal local optima. As a result, these kinds of optimization problems are sensitive to the initial solution. Providing an appropriate initial solution can increase the likelihood of finding the globally optimal solution. In this paper, an initialization technique that uses the polynomial coefficients in a full, albeit low order, NARX model is proposed. This technique generates a tensor from the coefficients in the from full polynomial NARX model and applies a tensor factorization in order to generate an appropriate starting point for decoupled polynomial NARX model optimization problem. The proposed technique is applied to nonlinear benchmark problem and the results are promising.

Multi-objective evolutionary framework for non-linear system identification: A comprehensive investigation

The present study proposes a multi-objective framework for structure selection of nonlinear systems which are represented by polynomial NARX models. This framework integrates the key components of Multi-Criteria Decision Making (MCDM) which include preference handling, Multi-Objective Evolutionary Algorithms (MOEAs) and a posteriori selection. To this end, three well-known MOEAs such as NSGA-II, SPEA-II and MOEA/D are thoroughly investigated to determine if there exists any significant difference in their search performance. The sensitivity of all these MOEAs to various qualitative and quantitative parameters, such as the choice of recombination mechanism, crossover and mutation probabilities, is also studied. These issues are critically analyzed considering seven discrete-time and a continuous-time benchmark nonlinear system as well as a practical case study of non-linear wave-force modeling. The results of this investigation demonstrate that MOEAs can be tailored to determine the correct structure of nonlinear systems. Further, it has been established through frequency domain analysis that it is possible to identify multiple valid discrete-time models for continuous-time systems. A rigorous statistical analysis of MOEAs via performance sweet spots in the parameter space convincingly demonstrates that these algorithms are robust over a wide range of control parameters.

Using Decoupling Methods to Reduce Polynomial NARX Models

The polynomial NARX model, where the output is a polynomial function of past inputs and outputs, is a commonly used equation error model for nonlinear systems. While it is linear in the variables, which simplifies its identification, it suffers from two major drawbacks: the number of parameters grows combinatorially with the degree of the nonlinearity, and it is a black box model, which makes it difficult to draw any insights from the identified model. Polynomial decoupling techniques are used to replace the multiple-input single-output polynomial with a decoupled polynomial structure comprising a transformation matrix followed by bank of SISO polynomials, whose outputs are then summed. This approach is demonstrated on two benchmark systems: The Bouc-Wen friction model and the data from the Silverbox model. In both cases, the decoupling results in a substantial reduction in the number of parameters, and allows some insight into the nature of the nonlinearities in the system.

Influence of Sampling Rate and Discretization Methods in the Parameter Identification of Systems with Hysteresis

Hysteresis is a nonlinear behaviour, which has been considered very hard to model. It is commonly found in actuators and sensors, involving quasi-static memory effects between input and output variables. Usually, continuous time models are used to model this feature. However, polynomial NARX model has come up as an alternative to model this behaviour. Since NARX models are discrete-time models, it is important to verify how the sampling rate interfere in obtaining the mathematical model. Further, frequently continuous-time models are used as a bench test, to generate data for identification of several nonlinear behaviour, including hysteresis. This paper investigates how the sampling rate and discretization methods affects the parameter identification of a NARX model for a system with hysteresis. Improved Euler and fourth order Runge-Kutta methods are applied in a Bouc-Wen model for a magneto-rheological damper, which is used as a system to be identified by a NARX model, considering the above mentioned scenario. Least-square based technique is used in this work to estimate model parameters.

The Performance of Binary Artificial Bee Colony (BABC) in Structure Selection of Polynomial NARX and NARMAX Models

This paper explores the capability of the Binary Artificial Bee Colony (BABC) algorithm for feature selection of Nonlinear Autoregressive Moving Average with Exogenous Inputs (NARMAX) model, and compares its implementation with the Binary Particle Swarm Optimization (BPSO) algorithm. A binarized modification of the BABC algorithm was used to perform structure selection of the NARMAX model on a Flexible Robot Arm (FRA) dataset. The solution quality and convergence was compared with the BPSO optimization algorithm. Fitting and validation tests were performed using the One-Step Ahead (OSA), correlation and histogram tests. BABC was able to outperform BPSO in terms of convergence consistency with equal solution quality. Additionally, it was discovered that BABC was less prone to converge to local minima while BPSO was able to converge faster. Results from this study showed that BABC was better-suited for structure selection in huge dataset and the convergence has been proven to be more consistent relative to BPSO.

A randomised approach for NARX model identification based on a multivariate Bernoulli distribution

ABSTRACTThe identification of polynomial NARX models is typically performed by incremental model building techniques. These methods assess the importance of each regressor based on the evaluation of partial individual models, which may ultimately lead to erroneous model selections. A more robust assessment of the significance of a specific model term can be obtained by considering ensembles of models, as done by the RaMSS algorithm. In that context, the identification task is formulated in a probabilistic fashion and a Bernoulli distribution is employed to represent the probability that a regressor belongs to the target model. Then, samples of the model distribution are collected to gather reliable information to update it, until convergence to a specific model. The basic RaMSS algorithm employs multiple independent univariate Bernoulli distributions associated to the different candidate model terms, thus overlooking the correlations between different terms, which are typically important in the selection process. Here, a multivariate Bernoulli distribution is employed, in which the sampling of a given term is conditioned by the sampling of the others. The added complexity inherent in considering the regressor correlation properties is more than compensated by the achievable improvements in terms of accuracy of the model selection process.

Sparse Bayesian Identification of Polynomial NARX Models

In this paper a novel sparse Bayesian structure detection algorithm is introduced for the identification of nonlinear autoregressive with exogenous inputs (NARX) dynamic systems. The main advantage of this algorithm over alternatives is that parameter uncertainty is naturally incorporated, and parameter estimation by variational inference is computationally efficient, consisting of a sequence of closed form updates. The proposed framework is demonstrated through a commonly used simulated benchmark problem.

Application of the PLS–PH Method for Identifying Polynomial NARX Models

Nowadays, MPC controllers are widely applied in the industry, especially in chemical and petrochemical processes. In order to obtain good models for these controllers, a family of identification methods specific for them has been developed, namely the MPC relevant identification (MRI) methods. One of the algorithms of this family is the PLS–PH (partial least squares–prediction horizon), described in Lauri et al. (Chemometrics and Intelligent Laboratory Systems 100:118–126, 2010). The version of the MPC controllers based on nonlinear models, known as NMPC, has a much more restricted application, because they consist of more difficult optimization problems and present greater complexity for obtaining good nonlinear models for the process. In order to circumvent this difficulty, this article presents an extension of the family of MRI identification methods for obtaining models for NMPC controllers. This extension is accomplished by means of the PLS–PH algorithm for identifying models with nonlinear autoregressive with exogenous inputs (NARX) polynomial structure. An application of this method is also presented for identifying a nonlinear model based on data collected from an electric oven.

On the Relation between NARX Clusters and Even/Odd Nonlinearities through Frequency-Domain Analysis

Although polynomial NARX models have been intensively used in nonlinear system identification, few papers discussed how to relate the inner nonlinearities to specific types of clusters and regressors. The objective of this paper is to discuss this relationship for a class of systems that contain even or odd nonlinearities. This class covers block-structured models (Hammerstein, Wiener, and others) and systems with dynamic nonlinearities. To achieve the paper’s aim, a deep frequency-domain analysis is performed. For each type of nonlinearity, all the NARX clusters are investigated and the results show that each regressor type provides specific nonlinear contribution. The investigation is based on an output power spectra analysis when a specific multisinusoidal excitation is applied. According to the spectral contributions in some of the frequency lines, the nonlinearity classification is possible. By applying the same procedure to the clusters, one interprets how these clusters can (or not) contribute to explain the system nonlinearity. The paper findings have two major impacts: (i) one gains deep knowledge on how the nonlinearities are coded by the clusters, and (ii) this information can be used, for instance, to aid a structure selection procedure (ERR, term clustering, etc.) during the discarding of the clusters which are not able to explain the system nonlinear behavior. Some practical and experimental aspects are discussed, while numerical examples are presented to show the validity of the theoretical analysis.

A genetic programming approach based on Lévy flight applied to nonlinear identification of a poppet valve

Genetic programming (GP) is an evolutionary algorithm-based paradigm inspired by natural evolution to find a generalized hierarchy computer program description. GP adopts a tree-structured code to describe an identification problem. This paper proposed a GP method based on Lévy flight to estimate discrete polynomial NARX (Nonlinear AutoRegressive with eXogenous inputs) models. The Lévy flight random walks on increments distributed according to a heavy-tailed probability distribution formed by the α-stable distribution family. Besides, Lévy flight is a Markov processes. The distance from the origin of the random walk tends to a stable distribution after a large number of steps. These sorts of movements describe not only the fluctuations in share prices, but also natural behaviors as the way in which albatrosses search for food or the flight of many insects. In this paper, the contribution of Lévy flight is related to the tune of crossover and mutation probabilities in GP. The proposed GP method based on Lévy flight is utilized in an experimental application, a poppet valve. Poppet-type of valve is normally used in combustion engines to open and close intake and exhaust ports in the cylinder head. The very well machined adjust between seat and poppet face gives the sealing feature that is improved every time that the pressure inside the cylinder rises up pushing the valve head against its seat. This type of device is also used in the automotive industry to control the emission levels on combustion engines by recirculating burned gases into the combustion chamber. Results are presented to demonstrate the utility of the proposed GP method based on Lévy flight as promising technique in NARX (Nonlinear AutoRegressive with eXogenous inputs) model identification of a poppet valve.

Improved Structure Detection For Polynomial NARX Models Using a Multiobjective Error Reduction Ratio

This paper addresses the problem of structure detection for polynomial NARX models. It develops MERR, a multiobjective extension of a methodology well-known as the error reduction ratio (ERR). It is shown that it is possible to choose terms which take into account dynamics of prediction error and other types of affine information, such as fixed points or static curve. Two examples are included to illustrate the proposed methodology. A numerical example shows that the technique is able to reconstruct the structure of a system, known a priori. The identification of a pilot DC–DC buck converter shows that the proposed approach is capable to find models valid over a wide range of operation points. In this latter example, MERR is compared with ERR in two forms: (i) affine information is applied only in the structure selection for MERR and (ii) affine information is applied for structure selection for MERR and for parameter estimation for both MERR and ERR. In both comparisons, MERR presented nondominated solutions of Pareto set.

Identification of a Hammerstein model for an aerospace electrohydraulic servovalve

This paper discusses the black-box identification of a real aerospace electrohydraulic actuator. A Hammerstein model structure is adopted due to its simplicity and capacity to represent a wide set of nonlinear phenomena. The identification strategy begins with a frequency-domain analysis so that a polynomial NARX model structure can be defined. Then, an extended least squares procedure is applied to obtain the coefficient values.

Design and Evaluation of Control-Oriented Diesel Oxidation Catalyst Models

This paper presents different strategies for Diesel Oxidation Catalyst (DOC) modeling. Starting from the commonly used approach of describing the chemical and energy balances by first principles and fitting a small set of parameters often unknown, special emphasis is placed on data driven DOC modeling. Considering black as well as grey box approaches it is shown that the data driven methods provide a better model quality and, in the specific case, are easier to fit since the considered model class are polynomial NARX models, which are linear in parameters.

An efficient identification scheme for a nonlinear polynomial NARX model

Nonlinear polynomial NARX (nonlinear autoregressive with exogenous inputs) model identification often faces the problem of the huge size of the candidate pool, which makes the "wrapper" structure s...

Environmental Over-Threshold Event Forecasting using NARX Models

AbstractOver-threshold event forecasting is of paramount importance in the monitoring of environmental variables, such as those related to air pollution. This paper explores the use of nonlinear polynomial NARX models for the prediction of ozone concentration data, using specific cost functions and identification algorithms devised to enhance the prediction accuracy at the peak values of the signal, in order to improve the over-threshold event detection. Some preliminary results of the experimental data analysis carried out on observed time histories are illustrated to show the effectiveness of the presented methodology.

Adaptive model selection for polynomial NARX models

Two algorithms are proposed for the adaptive model selection of polynomial non-linear autoregressive with exogenous variable (NARX) models. The recursive forward regression with pruning (RFRP) algorithm is based on a recursive orthogonal least-squares (ROLS) procedure and efficiently integrates model augmentation and pruning to reduce processing time whenever new data are available. The algorithm provides excellent model structure tracking compared to different OLS-based model selection policies. A less accurate but much faster algorithm that can be used for time-critical applications is the ROLS-LASSO. This algorithm uses a recursive version of the least absolute shrinkage and selection operator (LASSO) regularisation approach for structure selection. It features a recursive standardisation of the regressors and performs parameter estimation with ROLS. A sliding window data updating is here adopted for both algorithms, although the methods seamlessly generalise to exponential windowing with forgetting factor. Some simulation examples are provided to demonstrate the model tracking capabilities of the algorithms.

Iterative Identification of Polynomial NARX Models for Complex Multi-Input Systems

This work proposes an iterative identification algorithm for polynomial NARX models with several inputs. An optimal input design procedure is used to obtain persistent excitation for all regressors taking into account actuator and safety constraints. Through this procedure and the iterative increase of the polynomial degree, systems of unknown nonlinear structure can be approximated. The paper presents the procedure and results using a numerical and experimental example – the air-path of a combustion engine.

Polynomial NARX Model Identification: a Wiener–Hammerstein Benchmark

This work discusses the performance of several NARX model identification techniques on a benchmark problem for black–box identification methods proposed in (Schoukens et al., 2008), concerning a SISO electronic nonlinear system with a Wiener–Hammerstein structure, originally documented in (Vandersteen, 1997). The objective being the obtainment of an accurate simulation model, capable of replicating the dynamic behavior of the system without using past measured output data, both prediction error and simulation error minimization methods have been tested and compared.