Nonlinear Autoregressive Neural Network Research Articles

Energy Consumption Prediction in a Novel Automated Photovoltaic Design Platform

This paper describes a multi-step algorithm used to predict and typify the energy consumption profile of a prosumer, allowing the automation of the design of self-consumption photovoltaic (PV) power systems in a novel platform called PV SPREAD. The algorithm uses different methodologies to address various possible scenarios of data availability. In this paper, those scenarios are addressed using nonlinear autoregressive artificial neural networks (ANN) with external inputs (NARX) to predict energy consumption. Results reveal that the proposed algorithm successfully addresses data gaps in a hotel load profile used as a case study. The results also show the limitations of NARX when residential clients are analyzed.

An Effective Hybrid NARX-LSTM Model for Point and Interval PV Power Forecasting

This paper proposes an effective Photovoltaic (PV) Power Forecasting (PVPF) technique based on hierarchical learning combining Nonlinear Auto-Regressive Neural Networks with exogenous input (NARXNN) with Long Short-Term Memory (LSTM) model. First, the NARXNN model acquires the data to generate a residual error vector. Then, the stacked LSTM model, optimized by Tabu search algorithm, uses the residual error correction associated with the original data to produce a point and interval PVPF. The performance of the proposed PVPF technique was investigated using two real datasets with different scales and locations. The comparative analysis of the NARX-LSTM with twelve existing benchmarks confirms its superiority in terms of accuracy measures. In summary, the proposed NARX-LSTM technique has the following major achievements: 1) Improves the prediction performance of the original LSTM and NARXNN models; 2) Evaluates the uncertainties associated with point forecasts with high accuracy; 3) Provides a high generalization capability for PV systems with different scales. Numerical results of the comparison of the proposed NARX-LSTM method with two real-world PV systems in Australia and USA demonstrate its improved prediction accuracy, outperforming the benchmark approaches with an overall normalized Rooted Mean Squared Error (nRMSE) of 1.98% and 1.33% respectively.

Water Quality Prediction Using Artificial Intelligence Algorithms.

During the last years, water quality has been threatened by various pollutants. Therefore, modeling and predicting water quality have become very important in controlling water pollution. In this work, advanced artificial intelligence (AI) algorithms are developed to predict water quality index (WQI) and water quality classification (WQC). For the WQI prediction, artificial neural network models, namely nonlinear autoregressive neural network (NARNET) and long short-term memory (LSTM) deep learning algorithm, have been developed. In addition, three machine learning algorithms, namely, support vector machine (SVM), K-nearest neighbor (K-NN), and Naive Bayes, have been used for the WQC forecasting. The used dataset has 7 significant parameters, and the developed models were evaluated based on some statistical parameters. The results revealed that the proposed models can accurately predict WQI and classify the water quality according to superior robustness. Prediction results demonstrated that the NARNET model performed slightly better than the LSTM for the prediction of the WQI values and the SVM algorithm has achieved the highest accuracy (97.01%) for the WQC prediction. Furthermore, the NARNET and LSTM models have achieved similar accuracy for the testing phase with a slight difference in the regression coefficient (RNARNET = 96.17% and RLSTM = 94.21%). This kind of promising research can contribute significantly to water management.

SELECTION OF INPUT VARIABLES OF NONLINEAR AUTOREGRESSIVE NEURAL NETWORK MODEL FOR TIME SERIES DATA FORECASTING

NARNN is a type of ANN model consisting of a limited number of parameters and widely used for various applications. This study aims to determine the appropriate NARNN model, for the selection of input variables of nonlinear autoregressive neural network model for time series data forecasting, using the stepwise method. Furthermore, the study determines the optimal number of neurons in the hidden layer, using a trial and error method for some architecture. The NARNN model is combined in three parts, namely the learning method, the activation function, and the ensemble operator, to get the best single model. Its application in this study was conducted on real data, such as the interest rate of Bank Indonesia. The comparison results of MASE, RMSE, and MAPE values with ARIMA and Exponential Smoothing models shows that the NARNN is the best model used to effectively improve forecasting accuracy.

Model Time Series untuk Prediksi Jumlah Kasus Infeksi Coronavirus (Covid-19) di Sulawesi Selatan

Since it was declared a pandemic outbreak, the COVID 19 virus has become one of the main focuses of countries in the world in efforts to prevent the spread of the virus, including Indonesia. The areas of greatest severity in Indonesia include Jakarta, East Java, West Java and South Sulawesi. South Sulawesi Province is recorded as the largest area exposed to the COVID 19 pandemic outside Java Island. Predicting the number of COVID 19 cases is an alternative in preventing the spread through making government policies based on predictive data. This article presents a predictive model for the number of COVID 19 cases based on the ARIMA, Holt Winters and Nonlinear Autoregressive Neural Network (NAR-NN) Model. The results of the analysis show that the ARIMA Model (1,1,1) has a better level of prediction accuracy than the HW and NAR-NN models based on the MAPE criteria. Meanwhile, for the RMSE, MAE and MPE criteria, the NAR-NN model is better than others.

Motion Prediction of Human Wearing Powered Exoskeleton

With the development of powered exoskeleton in recent years, one important limitation is the capability of collaborating with human. Human-machine interaction requires the exoskeleton to accurately predict the human motion of the upcoming movement. Many recent works implement neural network algorithms such as recurrent neural networks (RNN) in motion prediction. However, they are still insufficient in efficiency and accuracy. In this paper, a Gaussian process latent variable model (GPLVM) is employed to transform the high-dimensional data into low-dimensional data. Combining with the nonlinear autoregressive (NAR) neural network, the GPLVM-NAR method is proposed to predict human motions. Experiments with volunteers wearing powered exoskeleton performing different types of motion are conducted. Results validate that the proposed method can forecast the future human motion with relative error of 2%∼5% and average calculation time of 120 s∼155 s, depending on the type of different motions.

Are Neural Network Models Truly Effective at Forecasting? An Evaluation of Forecast Performance of Traditional Models with Neural Network Model for the Macroeconomic Data of G-7 Countries

Forecasting macroeconomic and financial data are always difficult task to the researchers. Various statisticaland econometrics techniques have been used to forecast these variables more accurately. Furthermore, in the presenceof structural break, linear models are failed to model and forecast. Therefore, this study examines the forecastingperformance of economic variables of G7 countries: France, Italy, Canada, Germany, Japan, United Kingdom andUnited States of America using non-linear autoregressive neural network (ARNN) model, linear auto regressive (AR)and Auto regressive integrated moving average model (ARIMA) models. The economic variables are inflation,exchange rate and Gross Domestic Product (GDP) growth for the period from 1970 to 2015. To measure theperformance of the considered model Root, Mean Square Error, Mean Absolute Error and Mean Absolute PercentageError are used. The results show that the forecasts from the non-linear neural network model are undoubtedly better ascompared to the AR and the Box–Jenkins ARIMA models.

Smart Climate Hydropower Tool: A Machine-Learning Seasonal Forecasting Climate Service to Support Cost–Benefit Analysis of Reservoir Management

This study proposes a climate service named Smart Climate Hydropower Tool (SCHT) and designed as a hybrid forecast system for supporting decision-making in a context of hydropower production. SCHT is technically designed to make use of information from state-of-art seasonal forecasts provided by the Copernicus Climate Data Store (CDS) combined with a range of different machine learning algorithms to perform the seasonal forecast of the accumulated inflow discharges to the reservoir of hydropower plants. The machine learning algorithms considered include support vector regression, Gaussian processes, long short-term memory, non-linear autoregressive neural networks with exogenous inputs, and a deep-learning neural networks model. Each machine learning model is trained over past decades datasets of recorded data, and forecast performances are validated and evaluated using separate test sets with reference to the historical average of discharge values and simpler multiparametric regressions. Final results are presented to the users through a user-friendly web interface developed from a tied connection with end-users in an effective co-design process. Methods are tested for forecasting the accumulated seasonal river discharges up to six months in advance for two catchments in Colombia, South America. Results indicate that the machine learning algorithms that make use of a complex and/or recurrent architecture can better simulate the temporal dynamic behaviour of the accumulated river discharge inflow to both case study reservoirs, thus rendering SCHT a useful tool in providing information for water resource managers in better planning the allocation of water resources for different users and for hydropower plant managers when negotiating power purchase contracts in competitive energy markets.

Driver Behavior Modeling Using Game Engine and Real Vehicle: A Learning-Based Approach

As a good example of Advanced Driver-Assistance Systems (ADAS), Advisory Speed Assistance (ASA) helps improve driving safety and possibly energy efficiency by showing advisory speed to the driver of an intelligent vehicle. However, driver-based speed tracking errors often emerge, due to the perception and reaction delay, as well as imperfect vehicle control, degrading the effectiveness of ASA system. In this study, we propose a learning-based approach to modeling driver behavior, aiming to predict and compensate for the speed tracking errors in real time. Subject drivers are first classified into different types according to their driving behaviors using the k-nearest neighbors (k-NN) algorithm. A nonlinear autoregressive (NAR) neural network is then adopted to predict the speed tracking errors generated by each driver. A specific traffic scenario has been created in a Unity game engine-based driving simulator platform, where ASA system provides advisory driving speed to the driver via a head-up display (HUD). A human-in-the-loop simulation study is conducted by 17 volunteer drivers, revealing a 53% reduction in the speed error variance and a 3% reduction in the energy consumption with the compensation of the speed tracking errors. The results are further validated by a field implementation with a real passenger vehicle.

Nexus between Energy Usability, Economic Indicators and Environmental Sustainability in Four ASEAN Countries: A Non-Linear Autoregressive Exogenous Neural Network Modelling Approach

This study investigates the use of a non-linear autoregressive exogenous neural network (NARX) model to investigate the nexus between energy usability, economic indicators, and carbon dioxide (CO2) emissions in four Association of South East Asian Nations (ASEAN), namely Malaysia, Thailand, Indonesia, and the Philippines. Optimized NARX model architectures of 5-29-1, 5-19-1, 5-17-1, 5-13-1 representing the input nodes, hidden neurons and the output units were obtained from the series of models configured. Based on the relationship between the input variables, CO2 emissions were predicted with a high correlation coefficient (R) > 0.9. and low mean square errors (MSE) of 3.92 × 10−21, 4.15 × 10−23, 2.02 × 10−19, 1.32 × 10−20 for Malaysia, Thailand, Indonesia, and the Philippines, respectively. Coal consumption has the highest level of influence on CO2 emissions in the four ASEAN countries based on the sensitivity analysis. These findings suggest that government policies in the four ASEAN countries should be more intensified on strategies to reduce CO2 emissions in relationship with the energy and economic indicators.

A new prediction approach of the COVID-19 virus pandemic behavior with a hybrid ensemble modular nonlinear autoregressive neural network.

We describe in this paper an approach for predicting the COVID-19 time series in the world using a hybrid ensemble modular neural network, which combines nonlinear autoregressive neural networks. At the level of the modular neural network, which is formed with several modules (ensembles in this case), the modules are designed to be efficient predictors for each country. In this case, an integrator is used to combine the outputs of the modules, in this way achieving the goal of predicting a set of countries. At the level of the ensembles, forming a part of the modular network, these are constituted by a set of modules, which are nonlinear autoregressive neural networks that are designed to be efficient predictors under particular conditions for each country. In each ensemble, the results of the modules are combined with an aggregator to achieve a better and improved result for the ensemble. Publicly available datasets of coronavirus cases around the globe from the last months have been used in the analysis. Interesting conclusions have been obtained that could be helpful in deciding the best strategies in dealing with this virus for countries in their fight against the coronavirus pandemic. In addition, the proposed approach could be helpful in proposing strategies for similar countries.

Online health diagnosis of lithium-ion batteries based on nonlinear autoregressive neural network

Battery health diagnostics is extremely crucial to guaranty the availability and reliability of the application in which they operate. Data-driven health diagnostics methods, particularly machine learning methods, have gained attention due to their simplicity and accuracy. However, a machine learning method is desired which can cope with the nonlinear behavior of battery cells and yet it avoids high computational complexity to work efficiently in online applications. The accuracy and robustness of machine learning methods strongly depend on the availability of a comprehensive battery degradation dataset that covers a variety of battery aging patterns. While many studies fail to address the aforementioned requirements, this study attempts to address them. Twenty-one nickel manganese cobalt oxide battery cells have been cycled in various operating conditions for more than two years to acquire the data. The partial charging voltage curve is explored to extract the health indicators that describe the health trajectory of the battery. Afterward, a nonlinear autoregressive exogenous (NARX) model is developed to capture the dependency between the health indicators and state of health of battery cells. Finally, the accuracy and robustness of the proposed method are validated. The results demonstrate the ability of NARX to health diagnosis of lithium-ion batteries with a maximum root mean squared error of 0.46 for untrained data. This indicates that the proposed model has high estimation accuracy, low computational complexity, and the ability of battery health estimation regardless of its aging pattern. These features point out the practicability of the proposed technique on online health diagnostics.

Assessing the Best Gap-Filling Technique for River Stage Data Suitable for Low Capacity Processors and Real-Time Application Using IoT.

Hydrometeorological data sets are usually incomplete due to different reasons (malfunctioning sensors, collected data storage problems, etc.). Missing data do not only affect the resulting decision-making process, but also the choice of a particular analysis method. Given the increase of extreme events due to climate change, it is necessary to improve the management of water resources. Due to the solution of this problem requires the development of accurate estimations and its application in real time, this work present two contributions. Firstly, different gap-filling techniques have been evaluated in order to select the most adequate one for river stage series: (i) cubic splines (CS), (ii) radial basis function (RBF) and (iii) multilayer perceptron (MLP) suitable for small processors like Arduino or Raspberry Pi. The results obtained confirmed that splines and monolayer perceptrons had the best performances. Secondly, a pre-validating Internet of Things (IoT) device was developed using a dynamic seed non-linear autoregressive neural network (NARNN). This automatic pre-validation in real time was tested satisfactorily, sending the data to the catchment basin process center (CPC) by using remote communication based on 4G technology.

Multistep short-term wind speed prediction using nonlinear auto-regressive neural network with exogenous variable selection

Precise wind speed prediction is a key factor in many energy applications, especially when wind energy is integrated with power grids. However, because of the intermittent and nonstationary nature of wind speed, modeling and predicting it is a challenge. In addition, using uncorrelated multivariate variables as exogenous input variables often adversely impacts the performance of prediction models. In this paper, we present a multistep short-term wind speed prediction using multivariate exogenous input variables. We implement different variable selection methods to select the best set of variables that significantly improve the performance of prediction models. We evaluate the performance of eight transfer learning methods, four shallow neural networks (NNs), and the persistence method on predicting the future values of wind speed using ultrashort-term, short-term, and multistep time horizons. We performed the evaluation over two-year high-sampled wind speed data averaged at 10-minute intervals. Results show that Nonlinear Auto-Regressive Exogenous (NARX) model outperformed all other methods, achieving an average mean absolute error (MAE) and root mean square error (RMSE) of 0.2205 and 0.3405 for multistep predictions, respectively. Despite the lower performance of the transfer learning methods (i.e., 0.43 and 0.58 for MAE and RMSE, respectively), it is believed that results could be further improved with a better enhancement of the feature selection and model parameters.

Deep learning-based forecasting model for COVID-19 outbreak in Saudi Arabia

COVID-19 outbreak has become a global pandemic that affected more than 200 countries. Predicting the epidemiological behavior of this outbreak has a vital role to prevent its spreading. In this study, long short-term memory (LSTM) network as a robust deep learning model is proposed to forecast the number of total confirmed cases, total recovered cases, and total deaths in Saudi Arabia. The model was trained using the official reported data. The optimal values of the model’s parameters that maximize the forecasting accuracy were determined. The forecasting accuracy of the model was assessed using seven statistical assessment criteria, namely, root mean square error (RMSE), coefficient of determination (R2), mean absolute error (MAE), efficiency coefficient (EC), overall index (OI), coefficient of variation (COV), and coefficient of residual mass (CRM). A reasonable forecasting accuracy was obtained. The forecasting accuracy of the suggested model is compared with two other models. The first is a statistical based model called autoregressive integrated moving average (ARIMA). The second is an artificial intelligence based model called nonlinear autoregressive artificial neural networks (NARANN). Finally, the proposed LSTM model was applied to forecast the total number of confirmed cases as well as deaths in six different countries; Brazil, India, Saudi Arabia, South Africa, Spain, and USA. These countries have different epidemic trends as they apply different polices and have different age structure, weather, and culture. The social distancing and protection measures applied in different countries are assumed to be maintained during the forecasting period. The obtained results may help policymakers to control the disease and to put strategic plans to organize Hajj and the closure periods of the schools and universities.

A learning-based resource provisioning approach in the fog computing environment

ABSTRACT With the recent advancements in distributed computing technologies, the fog computing model has emerged to provide resource capabilities at the edge of the network for executing IoT applications. However, due to the rapid growth of IoT applications and variability their workload over time, achieving an efficient resource provisioning solution to deal with time-varying workloads as one of the challenging tasks in resource management scope to be considered. In this work, we propose a learning-based resource provisioning approach for managing time-varying workloads of IoT applications in the fog network. Our proposed approach utilises the nonlinear autoregressive (NAR) neural network as prediction method and hidden Markov model (HMM) as a decision-maker to identify scaling decisions to provision the fog resources for serving of workloads of IoT applications. The effectiveness of our proposed solution is evaluated using extension experiments under real-world datasets, and the obtained results from iFogSim toolkit demonstrated that it yields a reduction of the delay and cost and improves resource energy consumption compared with existing baseline mechanisms.

Long-term effects of outdoor air pollution on mortality and morbidity–prediction using nonlinear autoregressive and artificial neural networks models

The daily association between mortality and air pollution is alarming and there is consistent evidence that air pollution has short-term effects on mortality and respiratory morbidity. Accurate predictions of these health effects of air pollution are essential for efficient planning of various sectors related to economic performances as well as strategic management to improve human health and air quality. The main objectives of this research were to determine the long-term effects of air pollution on respiratory morbidity and mortality with the Dickey-Fuller test as well as develop accurate prediction of respiratory morbidity and mortality with nonlinear autoregressive and artificial neural network models. This study examined daily variations in respiratory mortality and morbidity attributed to air pollutants in Ahvaz for 9-yrs period. The results showed that nitrogen monoxide and carbon monoxide have significant effect on total respiratory mortality. The sensitivity analysis and ADF test showed that the other pollutants (NO2, SO2, O3, PM10) had no significant effect on the total respiratory morbidity and mortality rate. For the nonlinear autoregressive model, topology 2-10-1 with two input, including nitrogen oxide and carbon oxide, ten hidden layers were the best (MSE = 0.1 and R = 0.82) for predicting the total mortality rate. Artificial neural network and nonlinear autoregressive models are very powerful methods for accurate prediction of respiratory mortality and mobility with at least three inputs. These findings strongly support the need for policymakers to set targets to reduce carbon monoxide and nitrogen monoxide concentrations in the environment.

Forecasting of some greenhouse gases content trend in the air of the Russian Arctic region

Increasing the number of heat extremes is the global problem associated with the greenhouse gas (GHG) emissions. The cause-effect relationships between the increase in average temperature and the greenhouse gases contents in the atmosphere are not completely determined. The monitoring data of the gases (methane, water vapor, carbon monoxide and carbon dioxide) and meteorological conditions on the Russia Arctic area, Belyy Island in summer seasons 2015–2017 was used to investigate the cycles of the dynamic change in the variability of the GHG contents. The linear trend of the average values of the temperature or the GHG content in the summer seasons during the three years was not detected. The highest daily mean temperature (283K) in the summer season was corresponding to the hottest 2016. The daily mean GHG concentrations did not differ within the mean plus or minus standard deviation every year. Analysis of the time series of GHG concentrations showed that the methane and water vapor data associated with the temperature contain decade, week, three- and one-day components. The 24- and 72-h time intervals were taken into account to forecast of the seasonal dynamics of the fluctuations by non-linear autoregressive neural network model which provided high predictive accuracy. For a one-day forecast, the correlation coefficients between the predicted and observed GHG contents were near 0.9.

Use of meteorological parameters for forecasting scarlet fever morbidity in Tianjin, Northern China.

The scarlet fever incidence has increased drastically in recent years in China. However, the long-term relationship between climate variation and scarlet fever remains contradictory, and an early detection system is lacking. In this study, we aim to explore the potential long-term effects of variations in monthly climatic parameters on scarlet fever and to develop an early scarlet-fever detection tool. Data comprising monthly scarlet fever cases and monthly average climatic variables from 2004 to 2017 were retrieved from the Notifiable Infectious Disease Surveillance System and National Meteorological Science Center, respectively. We used a negative binomial multivariable regression to assess the long-term impacts of weather parameters on scarlet fever and then built a novel forecasting technique by integrating an autoregressive distributed lag (ARDL) method with a nonlinear autoregressive neural network (NARNN) based on the significant meteorological drivers. Scarlet fever was a seasonal disease that predominantly peaked in spring and winter. The regression results indicated that a 1°C increment in the monthly average temperature and a 1-h increment in the monthly aggregate sunshine hours were associated with 17.578% (95% CI 7.674 to 28.393%) and 0.529% (95% CI 0.035 to 1.025%) increases in scarlet fever cases, respectively; a 1-hPa increase in the average atmospheric pressure at a 1-month lag was associated with 12.996% (95% CI 9.972 to 15.919%) decrements in scarlet fever cases. Based on the model evaluation criteria, the best-performing basic and combined approaches were ARDL(1,0,0,1) and ARDL(1,0,0,1)-NARNN(5, 22), respectively, and this hybrid approach comprised smaller performance measures in both the training and testing stages than those of the basic model. Climate variability has a significant long-term influence on scarlet fever. The ARDL-NARNN technique with the incorporation of meteorological drivers can be used to forecast the future epidemic trends of scarlet fever. These findings may be of great help for the prevention and control of scarlet fever.

Deep Learning-Enabled Sparse Industrial Crowdsensing and Prediction

Mobile Crowdsensing (MCS) is a powerful sensing paradigm, which provides sufficient social data for cognitive analytics in industrial sensing, and industrial manufacturing. Considering the sensing costs, sparse MCS, as a variant, only senses the data in a few subareas, and then infers the data of unsensed subareas by the spatio-temporal relationship of the sensed data. Existing works usually assume that the sensed data are linearly spatiotemporal dependent, which cannot work well in real-world nonlinear systems, and thus, result in low data inference accuracy. Moreover, in many cases, users not only require inferring the current data, but also have an interest in predicting the near future, which can provide more information for users' decision making. Facing these problems, we propose a deep learning-enabled industrial sensing, and prediction scheme based on sparse MCS, which consists of two parts: matrix completion and future prediction. Our goal is to achieve high-precision prediction of future moments under the hypothesis of sparse historical data. To make full use of the sparse data for prediction, we first propose a deep matrix factorization method, which can retain the nonlinear temporal-spatial relationship, and perform high-precision matrix completion. In order to predict the subareas' data in several future sensing cycles, we further propose a nonlinear autoregressive neural network, and a stacked denoising autoencoder to obtain the temporal-spatial correlation between the data from different cycles or subareas. According to the results gained by experiments on four real-world industrial sensing datasets consisting of six typical tasks, it can be seen that the method in this article improves the accuracy of prediction using sparse data.