Nonlinear Autoregressive With Exogenous Research Articles

Partial derivative-based dynamic sensitivity analysis expression for non-linear auto regressive with exogenous (NARX) model[sbnd]case studies on distillation columns and model's interpretation investigation

Constructing the reliable dynamic sensitivity profile for the output variable using the machine learning model is a challenging task; however, the dynamic sensitivity trends are helpful to understand the impact of the input variables on the system's performance. In this paper, we have derived the partial-derivative approach-based sensitivity analysis expression for the non-linear auto regressive with exogenous (NARX) model for the first time. The engineering systems-based case studies, i.e., two distillation columns with five and ten stages, respectively are taken which are commonly found in the chemical processing plants. Two output variables, i.e., liquid composition in tray 2 and tray 4 (Y2 and Y4) of a five-stage distillation column, and liquid composition in tray 7 (Y7) of a ten-stage (higher) distillation column are modelled by NARX with respect to time, feed concentration (Xf) and feed flow rate (Lf). The dynamic sensitivity profiles of the output variables with respect to Xf and Lf for the two distillation columns are plotted by the derived partial derivative-based sensitivity expression on the NARX model. Furthermore, the forward difference method of sensitivity analysis (first principle method) is also applied on the ordinary differential equations of the distillation columns to compute the sensitivity values of the output variables. A good agreement in the dynamic sensitivity values of the output variables with respect to the input variables is found for the two sensitivity analysis techniques thereby demonstrating the effectiveness of the partial-derivative approach for the improved NARX's interpretability performance. This research presents the explicit partial-derivative based sensitivity analysis expression for the NARX model which can be utilised for time-series applications and can provide the insights about the model's interpretation performance.

Improved Set-point Tracking Control of an Unmanned Aerodynamic MIMO System Using Hybrid Neural Networks

Artificial neural networks (ANN), an Artificial Intelligence (AI) technique, are both bio-inspired and nature-inspired models that mimic the operations of the human brain and the central nervous system that is capable of learning. This paper is based on a system that optimizes the performance of an uncertain unmanned nonlinear Multi-Input Multi-Output (MIMO) aerodynamic plant called Twin Rotor MIMO System (TRMS). The pitch and yaw angles which are challenging to control and optimize in practice, are being used as the input to the Nonlinear Auto-Regressive with eXogenous (NARX) model, and eventually trained. The training features use the Matlab Deep Learning Toolbox. The NARX structure has its core in the neural networks’ architecture. Data is collected from the TRMS testbed which is used to train the network. ANN as a Hybrid intelligent control strategy of ANN in combination with Pattern Search and Genetic Algorithm, is then utilized to optimize the parameters of the neural networks. At the end it was validated, tested and the optimized system run in simulation and compared with other intelligent and conventional controllers, with the proposed controller outperforming them, giving a very fast tracking control, stable and optimal performance that satisfactorily met all our design requirements.

NARX Deep Convolutional Fuzzy System for Modelling Nonlinear Dynamic Processes

This paper presents a new approach for modelling nonlinear dynamic processes (NDP). It is based on a nonlinear autoregressive with exogenous (NARX) inputs model structure and a deep convolutional fuzzy system (DCFS). The DCFS is a hierarchical fuzzy structure, which can overcome the deficiency of general fuzzy systems when facing high dimensional data. For relieving the curse of dimensionality, as well as improving approximation performance of fuzzy models, we propose combining the NARX with the DCFS to provide a good approximation of the complex nonlinear dynamic behavior and a fast-training algorithm with ensured convergence. There are three NARX DCFS structures proposed, and the appropriate training algorithm is adapted. Evaluations were performed on a popular benchmark—Box and Jenkin’s gas furnace data set and the four nonlinear dynamic test systems. The experiments show that the proposed NARX DCFS method can be successfully used to identify nonlinear dynamic systems based on external dynamics structures and nonlinear static approximators.

Novel Model Based on Artificial Neural Networks to Predict Short-Term Temperature Evolution in Museum Environment.

The environmental microclimatic characteristics are often subject to fluctuations of considerable importance, which can cause irreparable damage to art works. We explored the applicability of Artificial Intelligence (AI) techniques to the Cultural Heritage area, with the aim of predicting short-term microclimatic values based on data collected at Rosenborg Castle (Copenhagen), housing the Royal Danish Collection. Specifically, this study applied the NAR (Nonlinear Autoregressive) and NARX (Nonlinear Autoregressive with Exogenous) models to the Rosenborg microclimate time series. Even if the two models were applied to small datasets, they have shown a good adaptive capacity predicting short-time future values. This work explores the use of AI in very short forecasting of microclimate variables in museums as a potential tool for decision-support systems to limit the climate-induced damages of artworks within the scope of their preventive conservation. The proposed model could be a useful support tool for the management of the museums.

Evaluation of Kriging-NARX Modeling for Uncertainty Quantification of Nonlinear SDOF Systems with Degradation

Structural assessment for collapse is commonly approached by observing the failure or collapse of systems fully incorporating degradation. Challenges however exist in the performance indicator or damage measure due to compound impacts of uncertainties of external (seismic excitation) and internal (structural properties) characteristics with degradation behavior. To account for the impacts of uncertainties, the state-of-the-art kriging nonlinear autoregressive with exogenous (NARX) model is explored in this study to replicate the response of nonlinear single-degree-of-freedom systems. The generalized hysteretic Bouc-Wen model with internal uncertainties is selected to emulate the stiffness and strength degradation. A probabilistic stochastic ground motion model is introduced to represent the external uncertainties. The global terms of NARX model are selected by least-angle regression algorithm and the kriging model is utilized to surrogate uncertain parameters into corresponding NARX model coefficients. The predictions of kriging NARX models are further compared with that of the polynomial chaos nonlinear autoregressive with exogenous input form model as well as Monte Carlo simulation. The comparisons show that kriging NARX model presents an effective and efficient meta-model technique for uncertainty quantification of systems with degradation.

Entropy Fuzzy System Identification for the Heave Flight Dynamics of a Model-Scale Helicopter

This article studies nonlinear system identification of a small scale and flybar-free unmanned helicopter, the Trex450 chopper, built using commercial off-the-shelf components. We employ the real-time input-output data, obtained from human-controlled flight tests, operating the aircraft under severe ground effects during the vertical flight maneuvers. We highlight the efficacy of the entropy fuzzy system identification method with respect to the performance of several well-known nonlinear system identification techniques (i.e., a Takagi-Sugeno Fuzzy system, an adaptive neuro-fuzzy inference system (ANFIS), and a nonlinear autoregressive with exogenous (NARX) model) as our benchmarks. Our research confirms the benefits of the entropy fuzzy identification technique. Despite being nonlinear, the proposed fuzzy model is relatively simple, transparent, and highly accurate to represent the complex non-linear dynamic behaviors of our unmanned helicopter under severe ground effects. Another major advantage of the proposed system identification technique is its ability to avoid overfitting, an essential requirement in modeling. Overall, the fuzzy system is also capable of achieving a delicate balance between maximizing the accuracy while minimizing the complexity of the acquired model.

Nonlinear model predictive control (NMPC) of the solvent-based post-combustion CO2 capture process

The flexible operation capability of solvent-based post-combustion capture (PCC) process is vital to efficiently meet the load variation requirement in the integrated upstream power plant. This can be achieved through the deployment of an appropriate control strategy. In this paper, a nonlinear model predictive control (NMPC) system was developed and analysed for the solvent-based PCC process. The PCC process was represented as a nonlinear autoregressive with exogenous (NARX) inputs model, which was identified through the forward regression with orthogonal least squares (FROLS) algorithm. The FROLS algorithm allows the selection of an accurate model structure that best describes the dynamics of the process. The simulation results showed that the NMPC gave better performance compared with linear MPC (LMPC) with an improvement of 55.3% and 17.86% for CO2 capture level control under the scenarios considered. NMPC also gave a superior performance for reboiler temperature control with the lowest ISE values. The results from this work will support the development and implementation of NMPC strategy on the PCC process with reduced computational time and burden.

A Kriging–NARX Model for Uncertainty Quantification of Nonlinear Stochastic Dynamical Systems in Time Domain

A novel approach, referred to as sparse Kriging–NARX (KNARX), is proposed for the uncertainty quantification of nonlinear stochastic dynamical systems. It combines the nonlinear autoregressive with exogenous (NARX) input model with the high fidelity surrogate model Kriging. The sparsity in the proposed approach is introduced in the NARX model by reducing the number of polynomial bases using the least-angle regression (LARS) algorithm. Sparse KNARX captures the nonlinearity of a problem by the NARX model, whereas the uncertain parameters are propagated using the Kriging surrogate model, and LARS makes the model efficient. The accuracy and the efficiency of the sparse KNARX was measured through uncertainty quantification applied to three nonlinear stochastic dynamical systems. The time-dependent mean and standard deviation were predicted for all the numerical examples. Instantaneous stochastic response characteristics and maximum absolute response were also predicted. All the results were compared with the full scale Monte Carlo simulation (MCS) results and a mean error was calculated for all the numerical problems to measure the accuracy. All the results had excellent agreement with the MCS results at a very limited computational cost. The efficiency of the sparse KNARX also was measured by the CPU time and the required number of surrogate model evaluations. In all instances, sparse KNARX outperformed other state-of-the-art methods, which justifies the applicability of this model for nonlinear stochastic dynamical systems.

Evaluation of Two Recurrent Neural Network Methods for Prediction of Irrigation Rate and Timing

HighlightsNARX and LSTM recurrent neural networks were evaluated for prediction of irrigation prescriptions.LSTM neural networks presented the best performance for irrigation scheduling using soil matric potential sensors.NARX neural networks had the best performance for predicting irrigation prescriptions using weather data.High performance for several time-ahead predictions using both recurrent neural networks, with R2 > 0.94.The results can be adopted as a decision-support tool in irrigation scheduling for fields with different types of soils.Abstract. The implementation of adequate irrigation strategies could be done through real-time monitoring of soil water status at several soil depths; however, this could also represent a complex nonlinear problem due to the plant-soil-weather relationships. In this study, two recurrent neural network (RNN) models were evaluated to estimate irrigation prescriptions. Data for this study were collected from an on-farm corn irrigation study conducted between 2017 and 2019 in Samson, Alabama. The study used hourly data of weather and soil matric potential (SMP) monitored at three soil depths from 13 sensor probes installed on a loamy fine sand soil and a sandy clay loam soil. Two neural network methods, i.e., a nonlinear autoregressive with exogenous (NARX) input system and long short-term memory (LSTM), were trained, validated, and tested with a maximum dataset of 20,052 records and a maximum of eight categorical attributes to estimate one-step irrigation prescriptions. The performance of both methods was evaluated by varying the model development parameters (neurons or blocks, dropout, and epochs) and determining their impact on the final model prediction. Results showed that both RNN models demonstrated good capability in the prediction of irrigation prescriptions for the soil types studied, with a coefficient of determination (R2) > 0.94 and root mean square error (RMSE) < 1.2 mm. The results of this study indicate that after training the RNNs using the dataset collected in the field, models using only SMP sensors at three soil depths obtained the best performance, followed by models that used only data of solar radiation, temperature, and relative humidity in the prediction of irrigation prescriptions. For future applicability, the RNN models can be extended using datasets from other places for training, which would allow the adoption of a unique data-driven soil moisture model for irrigation scheduling useful in a wide range of soil types. Keywords: Corn, Irrigation scheduling, Machine learning, Modeling, Soil matric potential sensor.

Multi-parametric modeling of water treatment plant using AI-based non-linear ensemble

AbstractIn this research, general regression neural network (GRNN), Hammerstein-wiener (HW) and non-linear autoregressive with exogenous (NARX) neural network, least square support vector machine (LSSVM) models were employed for multi-parametric (Hardness (mg/L), turbidity (Turb) (μs/cm), pH and suspended solid (SS) (mg/L)) modeling of Tamburawa water treatment plant (TWTP) at Kano, Nigeria. The weekly data from the TWTP were used and the predictive models were evaluated based on several numerical indicators. Four different non-linear ensemble techniques (GRNN-E, HW-E, NARX-E, and LSSVM-E) were subsequently employed. The comparison of the results of modeling showed that HW served as the best model for the simulation of Hardness, Turb, and SS while the NARX model demonstrated high capability in predicting pH. Yet, the HW and NARX as system identification techniques attained best overall predictive performance among the four modeling approaches. The HW model offers, therefore, a reliable and intelligent approach for predicting the treated Hardness, Turb, and SS and should be explored as a predictive tool in TWTP. Among the non-linear ensemble models, GRNN-E proved of high merit and increased the accuracy of the best single models significantly up to 30% for Hardness and Turb, 34% for pH, and 37% for SS with regards to the single models.

NARX Model Identification Using Correntropy Criterion in the Presence of Non-Gaussian Noise

In past years, the system identification area has emphasized the identification of nonlinear dynamic systems. In this field, polynomial nonlinear autoregressive with exogenous (NARX) models are widely used due to flexibility and prominent representation capabilities. However, the traditional identification algorithms used for model selection and parameter estimation with NARX models have some limitation in the presence of non-Gaussian noise, since they are based on second-order statistics that tightly depend on the assumption of Gaussianity. In order to solve this dependence, a novel identification method called simulation correntropy maximization with pruning (SCMP) based on information theoretic learning is introduced by this paper. Results obtained in non-Gaussian noise environment in three experiments (numerical, benchmark data set and measured data from a real plant) are presented to validate the performance of the proposed approach when compared to other similar algorithms previously reported in the literature, e.g., forward regression with orthogonal least squares and simulation error minimization with pruning. The proposed SCMP method has shown increased accuracy and robustness for three different experiments.

Machine Learning Algorithms for Short-Term Load Forecast in Residential Buildings Using Smart Meters, Sensors and Big Data Solutions

In this paper, we propose a scalable Big Data framework that collects the data from smart meters and weather sensors, pre-processes and loads it into a NoSQL database that is capable to store and further process large volumes of heterogeneous data. Then, a set of Machine Learning (ML) algorithms are designed and implemented to determine the load profiles and forecast the electricity consumption for residential buildings for the next 24 hours. For the Short-Term Load Forecast (STLF), a Feed-Forward Artificial Neural Network (FF-ANN) algorithm with backtracking adjustment of the learning rate that extends and optimizes the Nesterov learning method is proposed. Its performance is compared with six algorithms, i.e. FF-ANN with well-known learning methods, namely Momentum and Nesterov, Non-linear AutoRegressive with eXogenous (NARX), Deep Neural Network (DNN), Gradient Tree Boosting (GTB) and Random Forests (RF) that are competitive and powerful ML algorithms which have been successfully used for load forecast. Hence, for STLF, the seven algorithms are executed simultaneously and the best one is automatically selected considering its accuracy in terms of Root Mean Square Errors (RMSE). The proposed methodology contains the steps required to implement the Big Data framework, i.e. data pre-processing, transformation and loading, the configuration of the ML algorithms for dimensionality reduction, clustering, STLF with different algorithms from which the Best Performant Algorithm (BPA) is automatically selected to provide STLF for the next 24 hours. The methodology is ultimately tested considering a real case of a residential smart building.

Non-linear system identification of solvent-based post-combustion CO2 capture process

Solvent-based post combustion capture (PCC) is a well-developed technology for CO2 capture from power plants and industry. A reliable model that captures the dynamics of the solvent-based capture process is essential to implement suitable control design. Typically, first principle models are used, however they usually require comprehensive knowledge and deep understanding of the process. System identification approach is adopted to obtain a model that accurately describes the dynamics between key variables in the process. The nonlinear auto-regressive with exogenous (NARX) inputs model is employed to represent the relationship between the input variables and output variables as two Multiple-Input Single-Output (MISO) sub-models. The forward regression with orthogonal least squares (FROLS) algorithm is implemented to select an accurate model structure that best describes the dynamics within the process. The prediction performance of the identified NARX models is promising and shows that the models capture the underlying dynamics of the CO2 capture process.

Identification method for a class of periodic discrete-time dynamic nonlinear systems based on Sinusoidal ESN

Abstract In this paper, an improved identification method based on sinusoidal echo state network (SESN) is proposed to identify a class of periodic discrete-time dynamic nonlinear systems with or without noise. For periodic nonlinear systems, the sinusoidal state activation functions can provide more efficient mapping than the sigmoidal state activation functions. A matrix trace based online learning algorithm is constructed to train the output weights of SESN. Based on the Lyapunov stability theory, the asymptotical convergence of the identification error to zero is proved. A nonlinear autoregressive with exogenous (NARX) input model is used to validate the effectiveness of the proposed identification method.

Metamodeling of dynamic nonlinear structural systems through polynomial chaos NARX models

This paper introduces a metamodeling strategy able to account for the uncertainties in simulating nonlinear, dynamically evolving engineering systems. Polynomial Chaos (PC) expansion is implemented for the development of stochastic metamodels capable of representing the response of numerical models with uncertain input variables. The models employed are of the Nonlinear AutoRegressive with eXogenous (NARX) input form, comprising parameters that are random variables themselves. By expanding these parameters onto an appropriately selected PC basis, the resulting PC-NARX metamodel achieves vast reduction in computational time with sufficient accuracy, yielding a suitable tool for implementations where replacement of computationally costly models is sought.

English

This paper demonstrates the applicability of Artificial Neural Networks (ANNs) that use Multiple Back-Propagation networks (MBP) and Non-linear Autoregressive with Exogenous (NARX) for predicting the deflection of the smart wind turbine blade specimen. A neural network model has been developed to perform the deflection with respect to a number of wires required as the output parameter. The parameter includes load, current, time taken and deflection as input parameters. The network has been trained with experimental data obtained from experimental work. The various stages involved in the development of genetic algorithm based neural network model are addressed at length in this paper.

Comparison of NARX Neural Network and Classical Modelling Approaches

Classical optimization tools are effective when precise mechanistic models exist to support their design and implementation. However, most of the real-world processes are complex due to either nonlinearities or uncertainties (or both) and environmental variations, thus making realizing accurate mathematical models for such processes quite difficult or often impossible. Black box approach tends to present a better alternative in such situations. This paper presents a comparison of nonlinear autoregressive with eXogenous (NARX) neural network and traditional modelling techniques [autoregressive with exogenous input (ARX) and autoregressive moving average with exogenous input (ARMAX)]. The models were validated using experimental data from full-scale plants. Simulation results revealed that the performance of the NARX neural network is better compared to the ARMAX and ARX. The NARX neural network may serve as a valuable forecasting tool for the plants.

A Nonlinear Generalization of Spectral Granger Causality

Spectral measures of linear Granger causality have been widely applied to study the causal connectivity between time series data in neuroscience, biology, and economics. Traditional Granger causality measures are based on linear autoregressive with exogenous (ARX) inputs models of time series data, which cannot truly reveal nonlinear effects in the data especially in the frequency domain. In this study, it is shown that the classical Geweke's spectral causality measure can be explicitly linked with the output spectra of corresponding restricted and unrestricted time-domain models. The latter representation is then generalized to nonlinear bivariate signals and for the first time nonlinear causality analysis in the frequency domain. This is achieved by using the nonlinear ARX (NARX) modeling of signals, and decomposition of the recently defined output frequency response function which is related to the NARX model.

A two‐step scheme for polynomial NARX model identification based on MOEA with prescreening process

AbstractPolynomial NARX (nonlinear autoregressive with exogenous) model identification has received considerable attention in last three decades. However, in a high‐order nonlinear system, it is very difficult to obtain the model structure directly even with state‐of‐art algorithms, because the number of candidate monomial terms is huge and increases drastically as the model order increases. Motivated by this fact, in this research, the identification is performed in two steps: firstly a prescreening process is carried out to select a reasonable number of important monomial terms based on two kinds of the importance indices. Then, in the reduced searching space with only the selected important terms, multi‐objective evolutionary algorithm (MOEA) is applied to determine a set of significant terms to be included in the polynomial model with the help of independent validation data. In this way, the whole identification algorithm is implemented efficiently. Numerical simulations are carried out to demonstrate the effectiveness of the proposed identification method. © 2011 Institute of Electrical Engineers of Japan. Published by John Wiley & Sons, Inc.

Application of NARX neural networks in thermal dynamics identification of a pulsating heat pipe

The pulsating heat pipe (PHP) receiving much attention in industries is a novel type of cooling device. The distinguishing feature of PHPs is the unsteady flow oscillations formed by the passing non-uniform distributions of vapour plugs and liquid slugs. This study introduces a methodology of a non-linear auto-regressive with exogenous (NARX) neural network to analyze the thermal dynamics of a PHP in both the time and frequency domains. Three heating powers: 30, 70, and 110 W are tested, and all the predicted results are presented in quite good agreement with the measured results. Herein, the harmonic analysis of the non-linear structure can be equivalently conducted with generalized frequency response functions (GFRFs). Based on the non-linear coupling between the various input spectral components, the interpretations of the higher order GFRFs have been extensively presented for demonstrating the non-linear effects on the heat transfer of a PHP at different operating conditions.