Nonlinear Autoregressive Network Research Articles

A Combined Model and Data-Driven Approach for the Determination of Rotor Temperature in an Induction Machine.

The need for protection of electrical machines comes as a demand of safety regulations in the automotive industry as well as a result of the general desire to obtain a robust and reliable electric powertrain. This paper introduces a hybrid method for estimating the temperature of the rotor of an Induction Machine (IM) based on a Nonlinear Autoregressive Network with Exogenous inputs (NARX) used as a prediction function within a particle filter. The temperature of the stator case is measured, and the information is used as an input to a NARX network and as a variable to a thermal process with first-order dynamics which serves as an observation function. Uncertainties of the NARX and thermal model are determined and used to correct the posterior estimate. Experimental data are used from a real IM test-bench and the results prove the applicability and good performance.

Forecasting evapotranspiration in different climates using ensembles of recurrent neural networks

Accurate ahead evapotranspiration forecasting is crucial for irrigation planning, for wetlands, agricultural and forest habitats preservation, and for water resource management. Deep learning algorithms can be used to develop effective forecasting models of ahead evapotranspiration. In this study, three Recurrent Neural Network-based models were built for the prediction of short term ahead actual evapotranspiration. Two variants of each model were developed changing the employed algorithm, selecting between long short-term memory (LSTM) and nonlinear autoregressive network with exogenous inputs (NARX), while the modeling was performed in the context of an ensemble approach. The prediction models were trained and tested using data from two sites with different climates: Cypress Swamp, southern Florida, and Kobeh Valley, central Nevada. With reference to the subtropical climatic conditions of South Florida, LSTM models proved to be more accurate than NARX models, while some exogenous variables such as sensible heat flux and relative humidity did not affect the results significantly. An increase of the forecast horizon from 1 to 7 days resulted in a slight reduction in the accuracy of both the LSTM- and NARX based models. Considering instead the semi-arid climate of Central Nevada, NARX models generally provided more accurate results, which were only slightly affected by relative humidity, sensible heat flux, and forecast horizon. On the other hand, LSTM models performance decayed if sensible heat flux and relative humidity were neglected, and if the forecast horizon was increased from 1 to 7 days. Deep learning-based models can provide very accurate predictions of actual evapotranspiration, but the performance of the models can be significantly affected by local climatic conditions.

Modeling Wind Speed with a Long-Term Horizon and High-Time Interval with a Hybrid Fourier-Neural Network Model

The limited availability of local climatological stations and the limitations to predict the wind speed (WS) accurately are significant barriers to the expansion of wind energy (WE) projects worldwide. A methodology to forecast accurately the WS at the local scale can be used to overcome these barriers. This study proposes a methodology to forecast the WS with high-resolution and long-term horizons, which combines a Fourier model and a nonlinear autoregressive network (NAR). Given the nonlinearities of the WS variations, a NAR model is used to forecast the WS based on the variability identified with the Fourier analysis. The NAR modelled successfully 1.7 years of wind-speed with 3 hours of the time interval, what may be considered the longest forecasting horizon with high resolution at the moment.

Forecasting Model of Silicon Content in Molten Iron Using Wavelet Decomposition and Artificial Neural Networks

Silicon content forecasting models have been requested by the operational team to anticipate necessary actions during the blast furnace operation when producing molten iron, to control the quality of the product and reduce costs. This paper proposed a new algorithm to perform the silicon content time series up to 8 h ahead, immediately after the molten iron chemical analysis is delivered by the laboratory. Due to the delay of the laboratory when delivering the silicon content measurement, the proposed algorithm considers a minimum useful forecasting horizon of 3 h ahead. In a first step, it decomposes the silicon content time series into different subseries using the Maximal Overlap Discrete Wavelet Packet Transform (MODWPT). Next, all subseries forecasts were determined through Nonlinear Autoregressive (NAR) networks, and finally, these forecasts were summed to furnish the long-term forecast of silicon content. Using data from a real industry, we showed that the prediction error was within an acceptable range according to the blast furnace technical team.

Multivariate CDS risk premium prediction with SOTA RNNs on MI[N]T countries

In this study, CDS risk premiums of Mexico, Indonesia and Turkey were predicted by applying state-of-the-art forecasters in deep learning recurrent neural networks architectures which are the most recent ground-breaking predictors in the time series setting. The predictive power of each sota forecaster is compared, and the results are differentiated by country and type of sota predictors. While the long short-term memory model is better to predict Mexico’s CDS risk premiums, the nonlinear autoregressive network with exogenous inputs model is found to be more suitable for Indonesia and Turkey. The results of Turkey model reached the highest forecast accuracy.

Generating an Adaptive and Robust Walking Pattern for a Prosthetic Ankle-Foot by Utilizing a Nonlinear Autoregressive Network With Exogenous Inputs.

One of the major challenges in developing powered lower limb prostheses is emulating the behavior of an intact lower limb with different walking speeds over diverse terrains. Numerous studies have been conducted on control algorithms in the field of rehabilitation robotics to achieve this overarching goal. Recent studies on powered prostheses have frequently used a hierarchical control scheme consisting of three control levels. Most control structures have at least one element of discrete transition properties that requires numerous sensors to improve classification accuracy, consequently increasing computational load and costs. In this study, we proposed a user-independent and free-mode method for eliminating the need to switch among different controllers. We constructed a database by using four OPAL wearable devices (Mobility Lab, APDM Inc., USA) for seven able-bodied subjects. We recorded the gait of each subject at three ambulation speeds during ground-level walking to train a nonlinear autoregressive network with an exogenous input recurrent neural network (NARX RNN) to estimate foot orientation (angular position) in the sagittal plane using shank angular velocity as external input. The trained NARX RNN estimated the foot orientation of all the subjects at different walking speeds over flat terrain with an average root-mean-square error (RMSE) of 2.1° ± 1.7°. The minimum correlation between the estimated and measured values was 86%. Moreover, a t-test showed that the error was normally distributed with a high certainty level (0.88 minimum p -value).

Advanced Monitoring and Prediction of the Thermal State of Intelligent Battery Cells in Electric Vehicles by Physics-Based and Data-Driven Modeling

Novel intelligent battery systems are gaining importance with functional hardware on the cell level. Cell-level hardware allows for advanced battery state monitoring and thermal management, but also leads to additional thermal interactions. In this work, an electro-thermal framework for the modeling of these novel intelligent battery cells is provided. Thereby, a lumped thermal model, as well as a novel neural network, are implemented in the framework as thermal submodels. For the first time, a direct comparison of a physics-based and a data-driven thermal battery model is performed in the same framework. The models are compared in terms of temperature estimation with regard to accuracy. Both models are very well suited to represent the thermal behavior in novel intelligent battery cells. In terms of accuracy and computation time, however, the data-driven neural network approach with a Nonlinear AutoregRessive network with eXogeneous input (NARX) shows slight advantages. Finally, novel applications of temperature prediction in battery electric vehicles are presented and the applicability of the models is illustrated. Thereby, the conventional prediction of the state of power is extended by simultaneous temperature prediction. Additionally, temperature forecasting is used for pre-conditioning by advanced cooling system regulation to enable energy efficiency and fast charging.

Nonlinear generalized predictive controller based on ensemble of NARX models for industrial gas turbine engine

New design and operation of modern gas turbine engines (GTEs) are becoming more and more complex where several limitations and control modes should be fulfilled at the same time to accomplish a safe and ideal performance for the engine. For this purpose, a constrained multi-input multi-output (MIMO) non-linear model predictive controller (NMPC) based on neural network model is designed to fulfill the control requirements of a Siemens SGT-A65 three-spool aero-derivative gas turbine engine (ADGTE) used for power generation. However, the implementation of NMPC in real time has two challenges: Firstly, the design of an accurate non-linear model, which can run many times faster than real time. Secondly, the usage of a rapid and reliable optimization algorithm to solve the optimization problem in real time. To solve these issues, the constrained MIMO NMPC is created based on the generalized predictive control (GPC) algorithm as a result of its clarity, ease of use, and capacity to deal with problems in one algorithm. In addition, seven ensembles of eight multi-input single-output (MISO) non-linear autoregressive network with exogenous inputs (NARX) models are used as a base model for the GPC controller to predict the future process outputs. Estimation of free and forced responses of the GPC based on the neural network (NN) model of the plant each sampling time without performing instantaneous linearization is proposed in this study, which reduces the NMPC optimization problem to a linear optimization problem at each sampling step. In addition, the Hildreth's quadratic programming algorithm is used to solve the quadratic optimization problem within the NMPC controller, which offers ease of use and reliability in real time applications. To demonstrate the performance of the NNGPC controller developed in this study, we have compared the performance of the neural network generalized predictive control (NNGPC) controller to the existing controller of the SGT-A65 engine. The simulation results show that the NNGPC has demonstrated output responses with less oscillatory behavior and smoother control actions to the sudden variation in the electric load disturbance than those observed in the existing min-max controller. However, the min-max controller has faster response than that of the NNGPC controller.

Groundwater level forecasting with artificial neural networks: a comparison of long short-term memory (LSTM), convolutional neural networks (CNNs), and non-linear autoregressive networks with exogenous input (NARX)

Abstract. It is now well established to use shallow artificial neural networks (ANNs) to obtain accurate and reliable groundwater level forecasts, which are an important tool for sustainable groundwater management. However, we observe an increasing shift from conventional shallow ANNs to state-of-the-art deep-learning (DL) techniques, but a direct comparison of the performance is often lacking. Although they have already clearly proven their suitability, shallow recurrent networks frequently seem to be excluded from the study design due to the euphoria about new DL techniques and its successes in various disciplines. Therefore, we aim to provide an overview on the predictive ability in terms of groundwater levels of shallow conventional recurrent ANNs, namely non-linear autoregressive networks with exogenous input (NARX) and popular state-of-the-art DL techniques such as long short-term memory (LSTM) and convolutional neural networks (CNNs). We compare the performance on both sequence-to-value (seq2val) and sequence-to-sequence (seq2seq) forecasting on a 4-year period while using only few, widely available and easy to measure meteorological input parameters, which makes our approach widely applicable. Further, we also investigate the data dependency in terms of time series length of the different ANN architectures. For seq2val forecasts, NARX models on average perform best; however, CNNs are much faster and only slightly worse in terms of accuracy. For seq2seq forecasts, mostly NARX outperform both DL models and even almost reach the speed of CNNs. However, NARX are the least robust against initialization effects, which nevertheless can be handled easily using ensemble forecasting. We showed that shallow neural networks, such as NARX, should not be neglected in comparison to DL techniques especially when only small amounts of training data are available, where they can clearly outperform LSTMs and CNNs; however, LSTMs and CNNs might perform substantially better with a larger dataset, where DL really can demonstrate its strengths, which is rarely available in the groundwater domain though.

Improving Quality of Long-Term Bond Price Prediction Using Artificial Neural Networks

Purpose: The aim of this paper is to propose nonlinear autoregressive neural network which can improve quality of bond price forecasting. Methodology/Approach: Due to the complex nature of market information that influence bonds, artificial intelligence could be accurate, robust and fast choice of bond price prediction method. Findings: Our results have reached a coefficient of determination higher than 95% in the training, validation and testing sets. Moreover, we proposed the nonlinear autoregressive network with external inputs using 50 year interest-rate swaps denominated in EUR and volatility index VIX as two external variables. Research Limitation/Implication: Our sample of daily prices between 4th January 2016 and 13th January 2021 (totally 1,270 trading days) suggest that both Levenberg-Marquardt and Scaled conjugate gradient learning algorithms achieved excellent results. Originality/Value of paper: Despite the fact that both learning algorithms achieved satisfying outcomes, implementation of an independent variable into the autoregressive neural network environment had no significant impact on prediction ability of the model.

Effect of Daily Forecasting Frequency on Rolling-Horizon-Based EMS Reducing Electrical Demand Uncertainty in Microgrids

Accurate forecasting is a crucial task for energy management systems (EMSs) used in microgrids. Despite forecasting models destined to EMSs having been largely investigated, the analysis of criteria for the practical execution of this task, in the framework of an energy management algorithm, has not been properly investigated yet. On such a basis, this paper aims at exploring the effect of daily forecasting frequency on the performance of rolling-horizon EMSs devised to reduce demand uncertainty in microgrids by adhering to a reference planned profile. Specifically, the performance of a sample EMS, where the forecasting task is committed to a nonlinear autoregressive network with exogenous inputs (NARX) artificial neural network (ANN), has been studied under different daily forecasting frequencies, revealing a representative trend relating the forecasting execution frequency in the EMS and the reduction of uncertainty in the electrical demand. On the basis of such a trend, it is possible to establish how often is convenient to repeat the forecasting task for obtaining increasing performance of the EMS. The obtained results have been generalized by extending the analysis to different test scenarios, whose results have been found coherent with the identified trend.

Neural Networks Predicting Microbial Fuel Cells Output for Soft Robotics Applications.

The development of biodegradable soft robotics requires an appropriate eco-friendly source of energy. The use of Microbial Fuel Cells (MFCs) is suggested as they can be designed completely from soft materials with little or no negative effects to the environment. Nonetheless, their responsiveness and functionality is not strictly defined as in other conventional technologies, i.e. lithium batteries. Consequently, the use of artificial intelligence methods in their control techniques is highly recommended. The use of neural networks, namely a nonlinear autoregressive network with exogenous inputs was employed to predict the electrical output of an MFC, given its previous outputs and feeding volumes. Thus, predicting MFC outputs as a time series, enables accurate determination of feeding intervals and quantities required for sustenance that can be incorporated in the behavioural repertoire of a soft robot.

Reconstruction of the Radiation Belts for Solar Cycles 17–24 (1933–2017)

AbstractWe present a reconstruction of the dynamics of the radiation belts from solar cycles 17 to 24 which allows us to study how radiation belt activity has varied between the different solar cycles. The radiation belt simulations are produced using the Versatile Electron Radiation Belt (VERB)‐3D code. The VERB‐3D code simulations incorporate radial, energy, and pitch angle diffusion to reproduce the radiation belts. Our simulations use the historical measurements of Kp (available since solar cycle 17, i.e., 1933) to model the evolution radiation belt dynamics between L* = 1–6.6. A nonlinear auto regressive network with exogenous inputs (NARX) neural network was trained off GOES 15 measurements (January 2011–March 2014) and used to supply the upper boundary condition (L* = 6.6) over the course of solar cycles 17–24 (i.e., 1933–2017). Comparison of the model with long term observations of the Van Allen Probes and CRRES demonstrates that our model, driven by the NARX boundary, can reconstruct the general evolution of the radiation belt fluxes. Solar cycle 24 (January 2008–2017) has been the least active of the considered solar cycles which resulted in unusually low electron fluxes. Our results show that solar cycle 24 should not be used as a representative solar cycle for developing long term environment models. The developed reconstruction of fluxes can be used to develop or improve empirical models of the radiation belts.

Design and implementation of fault tolerant fractional order controllers for the output power of self-excited induction generator

In this paper, a static volt-ampere-reactive compensator (SVC) composed of fixed capacitor-thyristor controlled reactor (FC-TCR) is employed to regulate the terminal voltage of self-excited induction generator (SEIG) in the purpose of controlling the output power. In addition, the speed of the generator is adjusted by robust proportional-integral-derivative (PID) in order to regulate the frequency. Fractional order PID (FOPID) and Tilt-PID (T-PID) controllers are proposed to adjust the triggering angle of FC-TCR. The objectives of the paper are to design the optimal voltage controller with a robust PID frequency controller and to maintain the output power at a desired level in case of voltage sensor faults. In this regards, nonlinear autoregressive network with exogenous inputs (NARX) and small signal models of the SEIG are constructed where triggering angle and generator speed are inputs, whereas terminal voltage and frequency are outputs, respectively. Training of the NARX model is achieved with a high regression value (R2 = 0.99) where the accuracy of the small signal voltage model is 82%. After comparing the accuracy of the models, NARX-based fault detection architecture and fault tolerant controller are designed in order to avoid incorrect control of the output power. To establish the effectiveness of proposed fault tolerant controllers, simulation studies are conducted using particle swarm optimization (PSO). The designed controllers are performed experimentally on three-phase 5.5 kW, 400 V, 50 Hz, balanced loaded SEIG to confirm the effectiveness. The system is tested with proposed controllers under the healthy sensor, faulty sensor and dynamic reference conditions. Results give good agreement with simulation results and designed fault tolerant T-PID controller performs better tracking dynamics with small deviations whether the voltage sensor is faulty or not.

Quadrotor Attitude Dynamics Identification Based on Nonlinear Autoregressive Neural Network with Exogenous Inputs

ABSTRACT In the case of quadrotors, system identification is a challenging task because quadrotors are inherently unstable exhibit nonlinear behavior and significant coupling. In addition to this, quadrotors’ behavior is greatly influenced by characteristics and coefficients, which are very hard to measure directly or determine analytically. However, all the difficulties listed above are known to be successfully overcome by the use of artificial intelligence. In this paper, two system identification techniques were applied and compared to model quadrotor attitude dynamics. These techniques are Nonlinear Autoregressive Network with Exogenous Inputs (NARX) and continuous-time transfer function.

Experiment study of machine-learning-based approximate model predictive control for energy-efficient building control

The adoption of model predictive control (MPC) for building automation and control applications is challenged by the high hardware and software requirements to solve its optimization problem. This study proposes an approximate MPC that mimics the dynamic behaviours of MPC using the recurrent neural network with a structure of nonlinear autoregressive network with exogenous inputs. The approximate MPC is developed by learning from the measured operation data of buildings controlled by MPC, therefore it can produce MPC-like control for buildings without needing to solve the optimization problem, significantly reducing the computation load as compared to MPC. The proposed approximate MPC is implemented in two testbeds, an office and a lecture theatre, to control the air-conditioning systems. The control performance of the approximate MPC is compared to MPC as well as the original reactive control of the two testbeds. The approximate MPC retained most of the energy and thermal comfort performance of MPC in both testbeds. For the office, the MPC and approximate MPC reduced 58.5% and 51.6% of cooling energy consumption, respectively, as compared to the original control. For the lecture theatre, the MPC and approximate MPC reduced 36.7% and 36.2% of cooling energy consumption, respectively, as compared to the original control. Meanwhile, both approximate MPC and MPC significantly improved indoor thermal comfort in the two testbeds as compared to their original control. Despite having minor degradation in control performance the approximate MPC was more than 100 times faster than MPC in generating optimal control commands in each time step.

Prediction of wind speed profile using two artificial neural network models: an ab initio investigation in the Bapouh’s city, Cameroon

Purpose This paper aims to investigate the profile of the wind speed of a Cameroonian city for the very first time, as there is a growing trend for new wind energy installations in the West region of Cameroon. Two well-known artificial neural networks, namely, multi-layer perceptron (MLP) and nonlinear autoregressive network with exogenous inputs (NARX), were used to model the wind speed profile of the city of Bapouh in the West-region of Cameroon. Design/methodology/approach In this work, the profile of the wind speed of a Cameroonian city was investigated for the very first time since there is a growing trend for new wind energy installations in the West region of Cameroon. Two well-known artificial neural networks namely multi-layer perceptron (MLP) and nonlinear autoregressive network with exogenous inputs (NARX) were used to model the wind speed profile of the city of Bapouh in the West-region of Cameroon. The meteorological data were collected every 10 min, at a height of 50 m from the NASA website over a period of two months from December 1, 2016 to January 31, 2017. The performance of the model was evaluated using some well-known statistical tools, such as root mean square error (RMSE), mean absolute error (MAE) and mean absolute percentage error (MAPE). The input variables of the model were the mean wind speed, wind direction, maximum pressure, maximum temperature, time and relative humidity. The maximum wind speed was used as the output of the network. For optimal prediction, the influence of meteorological variables was investigated. The hyperbolic tangent sigmoid (Tansig) and linear (Purelin) were used as activation functions, and it was shown that the combination of wind direction, maximum pressure, maximum relative humidity and time as input variables is the best combination. Findings Maximum pressure, maximum relative humidity and time as input variables is the best combination. The correlation between MLP and NARX was computed. It was found that the MLP has the highest correlation when compared to NARX. Originality/value Two well-known artificial neural networks namely multi-layer perceptron (MLP) and nonlinear autoregressive network with exogenous inputs (NARX) were used to model the wind speed profile.

Battery lifetime prediction and performance assessment of different modeling approaches

SummaryLithium-ion battery technologies have conquered the current energy storage market as the most preferred choice thanks to their development in a longer lifetime. However, choosing the most suitable battery aging modeling methodology based on investigated lifetime characterization is still a challenge. In this work, a comprehensive aging dataset of nickel-manganese-cobalt oxide (NMC) cell is used to develop and/or train different capacity fade models to compare output responses. The assessment is conducted for semi-empirical modeling (SeM) approach against a machine learning model and an artificial neural network model. Among all, the nonlinear autoregressive network (NARXnet) can predict the capacity degradation most precisely minimizing the computational effort as well. This research work signifies the importance of lifetime methodological choice and model performance in understanding the complex and nonlinear Li-ion battery aging behavior.

Predicting protein secondary structure based on ensemble Neural Network

Protein structure prediction is very vital to innovative process of discovering new medications based on the knowledge of a biological target. It is also useful for scientifically exposing the biological basis of convoluted diseases and drug effects. Despite its usefulness, protein structure is very complex, thereby making its prediction to be arduous, timewasting and costly. These drawbacks necessitated the need to develop more effective techniques with high prediction capability. Conventional techniques for predicting protein structure are ineffective, perform poorly, expensive and slow. The reasons for these are due to the vague dissimilar sequences among protein structures, meaningless protein data, high dimensional data, and having to deal with highly imbalanced classification task. We proposed an Ensemble Neural Network learning model that consists of some Neural Network algorithms such as Feed Forward Neural Network (FFNN), Recurrent Neural Network (RNN), Cascade Forward Network (CFN) and Non-linear Autoregressive Network with Exogenous (NARX) models. These models were trained using training algorithms such as Levenberg-Marquardt (LM), Resilient Back Propagation (RBP) and Scaled Conjugate Gradient (SCG) to improve the performance. Experimental results show that our proposed model has superior performance compared to the other models compared.

Modelling Fuel Consumption andNOₓEmission of a Medium Duty Truck Diesel Engine With Comparative Time-Series Methods

This study focuses on different intelligent time series modelling techniques namely nonlinear autoregressive network with exogenous inputs (NARX), autoregressive integrated moving average with external inputs (ARIMAX), multiple linear regression (MLR), and regression error with autoregressive moving average (RegARMA), applied on a diesel engine to predict NO <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">x</sub> emission and fuel consumption. The experiment data are collected from a six cylinder, four stroke medium duty truck diesel engine, which is integrated on a passenger bus and operated in engine integration tests. NO <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">x</sub> emission and fuel consumption outputs are estimated with the help of input data; exhaust gas recirculation temperature and position, engine coolant temperature, engine speed, exhaust gas pressure, common rail pressure, intake manifold air temperature and pressure, accelerator pedal percentage, engine load, turbocharger variable geometry position and speed, and selective catalytic reduction outlet temperature. NARX artificial time series neural network, MLR, ARIMAX, and RegARMA time series techniques were separately applied for the estimation NO <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">x</sub> emission and fuel consumption outputs. The performance of the models is analyzed and evaluated with Bayesian information criterion (BIC) and root mean square error (RMSE) criteria. When the high cost and time loss of experimental testing are thought, using the intelligent modelling methodology provides far more accurate prediction and fast application abilities to analyze internal combustion engine dynamics for the control and calibration manner. As a result of the comparison of different types of modelling techniques, RegARMA technique comes to the forefront with 6707.6 BIC value with 105.58 RMSE for NO <sub xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">x</sub> emission model and 4026.4 BIC value with 7.93 RMSE for fuel consumption model.