Nonlinear ARMA Model Research Articles

Probabilistic seasonal forecasts of North Atlantic atmospheric circulation using complex systems modelling and comparison with dynamical models

AbstractDynamical seasonal forecast models are improving with time but tend to underestimate the amplitude of atmospheric circulation variability and to have lower skill in predicting summer variability than in winter. Here, we construct Nonlinear AutoRegressive Moving Average models with eXogenous inputs (NARMAX) to develop the analysis of drivers of North Atlantic atmospheric circulation and jet‐stream variability, focusing on the East Atlantic (EA) and Scandinavian (SCA) patterns as well as the North Atlantic Oscillation (NAO) index. New time series of these indices are developed from empirical orthogonal function (EOF) analysis. Geopotential height data from the ERA5 reanalysis are used to generate the EOFs. Sets of predictors with known associations with these drivers are developed and used to formulate a sliding‐window NARMAX model. This model demonstrates a high degree of predictive accuracy, as indicated by its average correlation coefficients over the testing period (2006–2021): 0.78 for NAO, 0.83 for EA and 0.68 for SCA. In comparison, the SEAS5 and GloSea5 dynamical forecast models exhibit lower correlations with observed circulation changes: for NAO, the correlation coefficients are 0.51 for SEAS5 and 0.34 for GloSea5, for EA they are 0.15 and 0.09, respectively, and for SCA, they are 0.28 and 0.24, respectively. Comparison of NARMAX predictions with forecasts and hindcasts from the SEAS5 and GloSea5 models highlights areas where NARMAX can be used to help improve seasonal forecast skill and inform the development of dynamical models, especially in the case of summer.

Modeling inflation rate factors on present consumption price index in Ethiopia: threshold autoregressive models approach

BackgroundInflation is the industrious and non-stop ascent in the overall prices of any given commodity in an economy. During the global food crisis, Ethiopia experienced an unprecedented increase in inflation ranked the highest in Africa. It is among the most macroeconomic variable described nonlinear behavior.ObjectiveThe main purpose of this study was intended to modeling inflation rate factors on present consumption price index (CPI) in Ethiopia: using the threshold autoregressive (TAR) models.MethodsThe study was utilized the secondary data collected from monthly data of CPI for inflation rate from January 1994 to December 2020 which was obtained from central statistical Agency. The forecast was applied between the nonlinear and linear ARMA models using different techniques. The unit root test of Dickey–Fuller test was made for each variables and applied lag length transformation for the variables that had unit root. A threshold autoregressive models was utilized for data handling technique using least square estimation.ResultsThe results showed that monthly rate of inflation was characterized a non-constant mean and an unstable variance. The outcome of Tsay tests was revealed that non linearity of CPI and SETAR(2,4,4) had the smallest value of AIC under this study. The forecasting performance comparison results were showed that the nonlinear SETAR model outperform the linear ARMA models. Moreover, the out-of-sample forecast indicates that the CPI of inflation has almost a constant trend. The in-sample forecast using the best-fit asymmetric for the SETAR(2,4,4) model the CPI series exhibits an upward trade until 2012; decreases until 2011; slightly increase up to 2018 and then decrease at the end of the study period.ConclusionThe superiority in performance of nonlinear models was attributed to their ability to capture the stochastic nature of the monthly rates as evident in the pattern of the forecast errors. The investigators are recommended that using TAR models policy makers can be able to capture the price volatility persistence and also forecasting can be made.

Hourly electricity price forecasting with NARMAX

Electricity price prediction through statistical and machine learning techniques captures market trends and would be a useful tool for energy traders to observe price fluctuations and increase their profits over time. A Nonlinear AutoRegressive Moving Average model with eXogenous inputs (NARMAX) identifies key energy-related factors that influence hourly electricity price through prediction modelling. We propose to use a transparent NARMAX model and analyse Irish Integrated Single Electricity Market (ISEM) data from May 2019 until April 2020 to determine which external factors have a significant impact on the electricity pricing. The experimental results indicate that historical electricity price, demand, and system generation are the most significant factors with historical electricity price being the most weighted factor and the largest Error Reduction Ratio (ERR). A NARMAX model generated using correlated lags was also considered to identify key energy-related lag factors that influence the electricity price. For justification, the significant lag factors are included as inputs in a Seasonal AutoRegressive Integrated Moving Average model with eXogenous input (SARIMAX) to determine if model performance improves with refinement. To conclude, using the NARMAX methodology with energy-related input factors helps to determine the significant factors and results in accurate predictions of electricity price.

Gaze Focalization System for Driving Applications Using OpenFace 2.0 Toolkit with NARMAX Algorithm in Accidental Scenarios.

Monitoring driver attention using the gaze estimation is a typical approach used on road scenes. This indicator is of great importance for safe driving, specially on Level 3 and Level 4 automation systems, where the take over request control strategy could be based on the driver’s gaze estimation. Nowadays, gaze estimation techniques used in the state-of-the-art are intrusive and costly, and these two aspects are limiting the usage of these techniques on real vehicles. To test this kind of application, there are some databases focused on critical situations in simulation, but they do not show real accidents because of the complexity and the danger to record them. Within this context, this paper presents a low-cost and non-intrusive camera-based gaze mapping system integrating the open-source state-of-the-art OpenFace 2.0 Toolkit to visualize the driver focalization on a database composed of recorded real traffic scenes through a heat map using NARMAX (Nonlinear AutoRegressive Moving Average model with eXogenous inputs) to establish the correspondence between the OpenFace 2.0 parameters and the screen region the user is looking at. This proposal is an improvement of our previous work, which was based on a linear approximation using a projection matrix. The proposal has been validated using the recent and challenging public database DADA2000, which has 2000 video sequences with annotated driving scenarios based on real accidents. We compare our proposal with our previous one and with an expensive desktop-mounted eye-tracker, obtaining on par results. We proved that this method can be used to record driver attention databases.

Data-Driven Stability Assessment of Multilayer Long Short-Term Memory Networks

Recurrent Neural Networks (RNNs) are increasingly being used for model identification, forecasting and control. When identifying physical models with unknown mathematical knowledge of the system, Nonlinear AutoRegressive models with eXogenous inputs (NARX) or Nonlinear AutoRegressive Moving-Average models with eXogenous inputs (NARMAX) methods are typically used. In the context of data-driven control, machine learning algorithms are proven to have comparable performances to advanced control techniques, but lack the properties of the traditional stability theory. This paper illustrates a method to prove a posteriori the stability of a generic neural network, showing its application to the state-of-the-art RNN architecture. The presented method relies on identifying the poles associated with the network designed starting from the input/output data. Providing a framework to guarantee the stability of any neural network architecture combined with the generalisability properties and applicability to different fields can significantly broaden their use in dynamic systems modelling and control.

Determination of Postural Control Mechanism in Overweight Adults Using The Artificial Neural Networks System and Nonlinear Autoregressive Moving Average Model

Determination of Postural Control Mechanism in Overweight Adults Using The Artificial Neural Networks System and Nonlinear Autoregressive Moving Average Model

A Novel Approach for Modeling Neural Responses to Joint Perturbations Using the NARMAX Method and a Hierarchical Neural Network.

The human nervous system is an ensemble of connected neuronal networks. Modeling and system identification of the human nervous system helps us understand how the brain processes sensory input and controls responses at the systems level. This study aims to propose an advanced approach based on a hierarchical neural network and non-linear system identification method to model neural activity in the nervous system in response to an external somatosensory input. The proposed approach incorporates basic concepts of Non-linear AutoRegressive Moving Average Model with eXogenous input (NARMAX) and neural network to acknowledge non-linear closed-loop neural interactions. Different from the commonly used polynomial NARMAX method, the proposed approach replaced the polynomial non-linear terms with a hierarchical neural network. The hierarchical neural network is built based on known neuroanatomical connections and corresponding transmission delays in neural pathways. The proposed method is applied to an experimental dataset, where cortical activities from ten young able-bodied individuals are extracted from electroencephalographic signals while applying mechanical perturbations to their wrist joint. The results yielded by the proposed method were compared with those obtained by the polynomial NARMAX and Volterra methods, evaluated by the variance accounted for (VAF). Both the proposed and polynomial NARMAX methods yielded much better modeling results than the Volterra model. Furthermore, the proposed method modeled cortical responded with a mean VAF of 69.35% for a three-step ahead prediction, which is significantly better than the VAF from a polynomial NARMAX model (mean VAF 47.09%). This study provides a novel approach for precise modeling of cortical responses to sensory input. The results indicate that the incorporation of knowledge of neuroanatomical connections in building a realistic model greatly improves the performance of system identification of the human nervous system.

Forecasting Conditional Variance of S&P100 Returns Using Feedforward and Recurrent Neural Networks

It is shown that time series about financial market variables are highly nonlinearly dependent on time. Fluctuations or volatility of returns on assets is one of them. Portfolio managers, option traders and market makers are all interested in volatility forecasting in order to get higher profits and less risky positions. The nonlinear dependence on time is very complex and parametric approaches, and linear models fail. Therefore as nonparametric tools artificial neural networks (ANNs) are candidates to deal with the volatility and/or return forecasting problems. On the other hand, based on the fact that volatility is time varying and that periods of high volatility tend to cluster, the most popular models in modeling volatility are GARCH type models because they can account excess kurtosis and asymmetric effects of financial time series. A standard GARCH(1,1) model usually indicates high persistence in the conditional variance, which may originate from structural changes. Hence it is natural that artificial neural networks (ANN) will be constructed to capture the nonlinear relationship between past return innovations and conditional variance which may be missed by linear regression models. First a usual feedforward, back propagation network is used. The structure of the return data makes FFANN difficult to converge. To overcome this difficulty a neural network with appropriate recurrent connections in the context of nonlinear ARMA models are used. These are the Jordan neural networks (JNN). Then Elman recurrent networks (ENN) and a mixture of the two (EJNN) are also used. The data set consists of returns of the S&P100 index daily closing prices obtained from the S&P100 website. The results indicate that the selected JNN(1,1,1) model has superior performances compared to the standard GARCH(1,1) model. The contribution of this paper can be seen in determining the appropriate NN that is comparable to the standard GARCH(1,1) model and its application in forecasting conditional variance of stock returns. Moreover, from the econometric perspective, NN models are used as a semi-parametric method that combines flexibility of nonparametric methods and the interpretability of parameters of parametric methods.

Split FTDNN-DFE Equalizer for Complex Non-linear Wireless Channels with NARMA Approximation

AbstractTime-dependent channel variations in wireless systems have been modeled using autoregressive (AR), moving average, and AR moving average (ARMA) approaches, which in certain conditions fail to provide the most suitable description of the scenarios. It lowers the quality of service, increases the dependence on the traditional tapped delay line model, and requires more reference symbols as fading in the channel increases. Non-linear attributes in channels are best described by non-linear AR (NAR) and non-linear ARMA (NARMA) models, which capture the channel state information (CSI) better. Learning-based systems like artificial neural networks (ANN) are better equipped to deal with NAR and NARMA approximations of channels, recover CSI, and provide proper channel estimation due to their natural association with non-linear computing. Though decision feedback equalizers (DFE) are the traditional and preferred approaches to reduce the ill effects of channel-induced variations, these are unable to mitigate noise-triggered responses that ANNs can. Here, DFEs and special forms of ANNs are combined to recover CSI from NAR and NARMA approximations of time-varying channels. We especially configure a class of fully focused time delay neural network designed in split form in unitary and composite modes with a DFE to track real and imaginary components of signals. Experimental results establish the effectiveness of such hybrid tools in tracking time-varying non-linear channels covering International Telecommunication Union pedestrian to vehicular conditions.

Nonlinear Mobile Link Adaptation Using Modified FLNN and Channel Sounder Arrangement

In a typical mobile environment, the varying speeds of transmit–receive pairs make traditional channel estimation methods inefficient due to continuously altering requirement of high density reference symbols. It has been largely instrumental in driving efforts to formulate innovative solutions, which are appropriate for such situations. Previously, with high computational cost, autoregressive moving average (ARMA) models of the stochastic wireless channels though appeared to be effective but could not efficiently incorporate the true nonlinearities observed in a practical situation. Therefore, nonlinear ARMA models based on artificial neural networks gained popularity. Yet certain challenges continue to exist, which are related to approximating all aspects of a real time situation, encompassing the non-linearities observed in a stochastic wireless channel, reducing training latency, enhancing processing capability, and deriving appropriate neuro-computational topologies. Modified functional link neural network with linearized activation function (FLNNLA) with nonlinear functional expansion is found to be more suitable for modeling stochastic wireless channels and removing the above-mentioned shortcomings. The proposed FLNNLA models the nonlinear tap gain process efficiently, reduces computational complexity, and enhances receiver performance with less learning cycles, better spectral efficiency and emerges as a strong candidate for being a part of upcoming receiver designs.

System identification of smart buildings under ambient excitations

This paper proposes a nonlinear autoregressive moving average (NARMA) model for use in system identification (SI) of high performance smart buildings under ambient excitations. The NARMA model is implemented by including the cross terms of output signals to a linear autoregressive moving average (LARMA) time series model. To demonstrate the effectiveness of the proposed NARMA approach, a three-story building equipped with smart control devices is investigated under a variety of ambient excitations. To access the robustness of the proposed model, it is tested under various levels of measurement noises. It is demonstrated from the extensive simulations that the proposed NARMA model is effective in predicting the ambient vibration responses of the high performance smart buildings with severe measurement noises.

A lower bound error for free-run simulation of the polynomial NARMAX

ABSTRACTA lower bound error for free-run simulation of the polynomial NARMAX (Nonlinear AutoRegressive Moving Average model with eXogenous input) is introduced. The ultimate goal of the polynomial NARMAX is to predict an arbitrary number of steps ahead. Free-run simulation is also used to validate the model. Although free-run simulation of the polynomial NARMAX is essential, little attention has been given to the error propagation to round off in digital computers. Our procedure is based on the comparison of two pseudo-orbits produced from two mathematical equivalent models, but different from the point of view of floating point representation. We apply successfully our technique for three identified models of the systems: sine map, Chua's circuit and Duffing–Ueda oscillator. This technique may be used to reject a simulation, if a required precision is greater than the lower bound error, increasing the numerical reliability in free-run simulation of the polynomial NARMAX.

Anomaly Detection of Network Traffic Based on Prediction and Self-Adaptive Threshold

Security problems with network are significant, such as network failures and malicious attacks. Monitoring network traffic and detect anomalies of network traffic is one of the effective manner to ensure network security. In this paper, we propose a hybrid method for network traffic prediction and anomaly detection. Specifically, the original network traffic data is decomposed into high-frequency components and low-frequency components. Then, non-linear model Relevance Vector Machine (RVM) model and ARMA (Auto Regressive Moving Average) model are employed respectively for prediction. After combining the prediction, a self-adaptive threshold method based on Central Limit Theorem (LCT) is introduced for anomaly detection. Moreover, our extensive experiments evaluate the efficiency of proposed method.

Recurrent-Neural-Network-Based Multivariable Adaptive Control for a Class of Nonlinear Dynamic Systems With Time-Varying Delay.

At the beginning, an approximate nonlinear autoregressive moving average (NARMA) model is employed to represent a class of multivariable nonlinear dynamic systems with time-varying delay. It is known that the disadvantages of robust control for the NARMA model are as follows: 1) suitable control parameters for larger time delay are more sensitive to achieving desirable performance; 2) it only deals with bounded uncertainty; and 3) the nominal NARMA model must be learned in advance. Due to the dynamic feature of the NARMA model, a recurrent neural network (RNN) is online applied to learn it. However, the system performance becomes deteriorated due to the poor learning of the larger variation of system vector functions. In this situation, a simple network is employed to compensate the upper bound of the residue caused by the linear parameterization of the approximation error of RNN. An e -modification learning law with a projection for weight matrix is applied to guarantee its boundedness without persistent excitation. Under suitable conditions, the semiglobally ultimately bounded tracking with the boundedness of estimated weight matrix is obtained by the proposed RNN-based multivariable adaptive control. Finally, simulations are presented to verify the effectiveness and robustness of the proposed control.

Output feedback adaptive control of a class of nonlinear discrete-time systems with unknown control directions and preceded by hysteresis

This paper considers the output feedback adaptive controller design problem for a class of discrete-time nonlinear systems in output feedback form with unknown control directions and preceded by unknown hysteresis. The problem of lacking in a-priori knowledge on the control directions and unknown hysteresis are solved by using the discrete Nussbaum gain and Prandtl–Ishlinskii model, respectively. The system is transformed into the form of a nonlinear auto regressive moving average (NARMA) model to construct an output feedback control. To overcome the noncausal problem in the control design, future output prediction laws and parameter update laws with the dead-zone technique are constructed on the basis of the NARMA model. The proposed control algorithm guarantees that all the signals in the controlled system are bound and the output tracking error is made to be neighbourhood around zero, ultimately. Finally, simulations are performed on a nonlinear system to show the effectiveness of the proposed method

GARCH based artificial neural networks in forecasting conditional variance of stock returns

Portfolio managers, option traders and market makers are all interested in volatility forecasting in order to get higher profits or less risky positions. Based on the fact that volatility is time varying in high frequency data and that periods of high volatility tend to cluster, the most popular models in modelling volatility are GARCH type models because they can account excess kurtosis and asymmetric effects of financial time series. A standard GARCH(1,1) model usually indicates high persistence in the conditional variance, which may originate from structural changes. The first objective of this paper is to develop a parsimonious neural networks (NN) model, which can capture the nonlinear relationship between past return innovations and conditional variance. Therefore, the goal is to develop a neural network with an appropriate recurrent connection in the context of nonlinear ARMA models, i.e., the Jordan neural network (JNN). The second objective of this paper is to determine if JNN outperforms the standard GARCH model. Out-of-sample forecasts of the JNN and the GARCH model will be compared to determine their predictive accuracy. The data set consists of returns of the CROBEX index daily closing prices obtained from the Zagreb Stock Exchange. The results indicate that the selected JNN(1,1,1) model has superior performances compared to the standard GARCH(1,1) model. The contribution of this paper can be seen in determining the appropriate NN that is comparable to the standard GARCH(1,1) model and its application in forecasting conditional variance of stock returns. Moreover, from the econometric perspective, NN models are used as a semi-parametric method that combines flexibility of nonparametric methods and the interpretability of parameters of parametric methods.

Recurrent support vector regression for a non-linear ARMA model with applications to forecasting financial returns

Motivated by recurrent neural networks, this paper proposes a recurrent support vector regression (SVR) procedure to forecast nonlinear ARMA model based simulated data and real data of financial returns. The forecasting ability of the recurrent SVR based ARMA model is compared with five competing models (random walk, threshold ARMA model, MLE based ARMA model, recurrent artificial neural network based ARMA model and feed-forward SVR based ARMA model) by using two forecasting accuracy evaluation metrics (NSME and sign) and robust Diebold---Mariano test. The results reveal that for one-step-ahead forecasting, the recurrent SVR model is consistently better than the benchmark models in forecasting both the magnitude and turning points, and statistically improves the forecasting performance as opposed to the usual feed-forward SVR.

Special issue on nonlinear modeling, estimation and control of biological systems

Dynamic modeling, state estimation and control are nowadays central techniques in the design, analysis, and optimization of bioprocesses in the industry, particularly with the increased importance of Process Analytical Technologies (PAT). The aim of this special issue is to account for some recent developments in the field by presenting a collection of ten articles written by distinguished colleagues who enthusiastically responded to our idea of dedicating a special issue to this subject. The first paper by David et al. develops a nitrogenbackboned model of wine-making in standard and nitrogen-added fermentations. Nitrogen has a strong impact on the key bio-mechanisms involved during the grape-must fermentation, but also on the synthesis of flavour markers determining the aromatic profile of the wine. This paper presents a dynamical mass balance model describing the main physiological phenomena involved in standard batch fermentations, i.e., consumption of sugar and nitrogen and synthesis of ethanol. It also includes nitrogen compounds such as hexose transporters. Moreover, a common practice in wine-making is the addition of nitrogen during the fermentation to boost and shorten the process duration. A tractable representation of this boost effect is, therefore, developed as an extension of the first model. It is apparent that yeast makes a different use of nitrogen depending on the fermentation stage at which the addition is effected, balancing the regrowth of biomass and the synthesis of supplementary hexose transporters. These models are validated in line with experimental evidence deduced from extensive experimental studies. Microalgae are often seen as a potential biofuel producer. To predict achievable productivities in the so-called raceway culturing system, the dynamics of photosynthesis has to be taken into account. In particular, the dynamical effect of inhibition by an excess of light (photoinhibition) must be represented. In the second paper by Hartmann et al., a model considering both photosynthesis and growth dynamics is proposed. This model involves three different time scales. The response of this model to fluctuating light with different frequencies is studied by slow–fast approximations. Therefore, three different regimes are identified, for which a simplified expression of the model can be derived. These expressions give a hint on productivity improvement, which can be expected by stimulating photosynthesis with a faster hydrodynamics. The lack of sensors for some relevant state variables in fermentation processes can be coped by developing appropriate software sensors. In the third paper by Acuna et al., various Artificial Neural Network (ANN) and Support Vector Machine (SVM) models are compared when acting as software sensors of biomass concentration for a Solid-Substrate Cultivation (SSC) process. In particular, Nonlinear AutoRegressive models with eXogenous variables (NARX) and Nonlinear AutoRegressive Moving Average models with eXogenous variables (NARMAX) are considered. Results show that NARMAX-SVM outperforms the other models with a symmetric mean absolute percentage error, also called SMAPE index, under 9 for a 20 % amplitude noise. In addition, NARMAX models perform better than NARX models under the same noise conditions because of their better predictive capabilities as J. A. Moreno Instituto de Ingenieria, Universidad Nacional Autonoma de Mexico, Cd. Universitaria, CP 04510 Mexico, D.F., Mexico e-mail: JMorenoP@ii.unam.mx

A Data-Driven Framework for Identifying Nonlinear Dynamic Models of Genetic Parts

A key challenge in synthetic biology is the development of effective methodologies for characterization of component genetic parts in a form suitable for dynamic analysis and design. In this investigation we propose the use of a nonlinear dynamic modeling framework that is popular in the field of control engineering but is novel to the field of synthetic biology: Nonlinear AutoRegressive Moving Average model with eXogenous inputs (NARMAX). The framework is applied to the identification of a genetic part BBa_T9002 as a case study. A concise model is developed that exhibits accurate representation of the system dynamics and a structure that is compact and consistent across cell populations. A comparison is made with a biochemical model, derived from a simple enzymatic reaction scheme. The NARMAX model is shown to be comparably simple but exhibits much greater prediction accuracy on the experimental data. These results indicate that the data-driven NARMAX framework is an attractive technique for dynamic modeling of genetic parts.

The stationarity and invertibility of a class of nonlinear ARMA models

We investigate some probabilistic properties of a new class of nonlinear time series models. A suffcient condition for the existence of a unique causal, strictly and weakly stationary solution is derived. To understand the proposed models better, we further discuss the moment structure and obtain some Yule-Walker difference equations for the second and third order cumulants, which can also be used for identification purpose. A suffcient condition for invertibility is also provided.