Nonlinear Neural Network Model Research Articles

Nexus between Energy Usability, Economic Indicators and Environmental Sustainability in Four ASEAN Countries: A Non-Linear Autoregressive Exogenous Neural Network Modelling Approach

This study investigates the use of a non-linear autoregressive exogenous neural network (NARX) model to investigate the nexus between energy usability, economic indicators, and carbon dioxide (CO2) emissions in four Association of South East Asian Nations (ASEAN), namely Malaysia, Thailand, Indonesia, and the Philippines. Optimized NARX model architectures of 5-29-1, 5-19-1, 5-17-1, 5-13-1 representing the input nodes, hidden neurons and the output units were obtained from the series of models configured. Based on the relationship between the input variables, CO2 emissions were predicted with a high correlation coefficient (R) > 0.9. and low mean square errors (MSE) of 3.92 × 10−21, 4.15 × 10−23, 2.02 × 10−19, 1.32 × 10−20 for Malaysia, Thailand, Indonesia, and the Philippines, respectively. Coal consumption has the highest level of influence on CO2 emissions in the four ASEAN countries based on the sensitivity analysis. These findings suggest that government policies in the four ASEAN countries should be more intensified on strategies to reduce CO2 emissions in relationship with the energy and economic indicators.

Forecasting of some greenhouse gases content trend in the air of the Russian Arctic region

Increasing the number of heat extremes is the global problem associated with the greenhouse gas (GHG) emissions. The cause-effect relationships between the increase in average temperature and the greenhouse gases contents in the atmosphere are not completely determined. The monitoring data of the gases (methane, water vapor, carbon monoxide and carbon dioxide) and meteorological conditions on the Russia Arctic area, Belyy Island in summer seasons 2015–2017 was used to investigate the cycles of the dynamic change in the variability of the GHG contents. The linear trend of the average values of the temperature or the GHG content in the summer seasons during the three years was not detected. The highest daily mean temperature (283K) in the summer season was corresponding to the hottest 2016. The daily mean GHG concentrations did not differ within the mean plus or minus standard deviation every year. Analysis of the time series of GHG concentrations showed that the methane and water vapor data associated with the temperature contain decade, week, three- and one-day components. The 24- and 72-h time intervals were taken into account to forecast of the seasonal dynamics of the fluctuations by non-linear autoregressive neural network model which provided high predictive accuracy. For a one-day forecast, the correlation coefficients between the predicted and observed GHG contents were near 0.9.

Learning What Makes Catalysts Good

Machine learning has proven a powerful tool for accelerating the computational characterization of energy materials. The neural-network approach by Lu et al. allows bypassing time-consuming first principles calculations for the design of catalysts based on high-entropy alloys. This work is an example of a growing number of case studies identifying descriptors of catalytic performance using machine learning instead of physical intuition.

A data-driven approach to estimate battery cell temperature using a nonlinear autoregressive exogenous neural network model

Abstract Battery cell temperature is a key parameter in battery life degradation, safety, and dynamic performance. Intense charging-discharging operations and high-ambient temperatures escalate battery cell temperature, which in turn accelerates its degradation. Therefore, accurate battery cell temperature estimation can play a significant role in ensuring the optimal operation of a battery energy storage system (BESS). In order to estimate battery cell temperature as accurate as possible, use of non-linear models is imperative due to the non-linear nature of the battery operation. This paper proposes a data-driven model based on a Non-linear Autoregressive Exogenous (NARX) neural network to estimate battery cell temperatures in a utility-scale BESS, considering strongly-correlated independent variables, e.g., charging-discharging current and ambient temperature. Due to different temperature and weather characteristics in each season, seasonal NARX models have also been derived and compared with the universal one. The proposed models’ performance has been verified using the field data collected from a grid-connected BESS within a PV plant. The simulation results show high accuracy of the proposed model compared to the measured data for both seasonal and universal models without considering the complexity of the large-scale battery and container thermal dynamics. In particular, in more than 95% of the time, the estimated values yield root mean squared errors (RMSE) below 1 ∘C in different conditions, which confirms the validity and accuracy of the proposed model. Moreover, seasonal models show better performance with 18% to 50% less RMSE on average (for 1 h to 24 h forward estimation) compared to the universal model.

Short-term prediction of passenger volume for urban rail systems: A deep learning approach based on smart-card data

Short-term prediction of passenger volume is a complex but critical task to urban rail companies, which desire prediction methods with high accuracy, time efficiency and good practicality. Good prediction results of the outbound passenger volume at urban rail stations are important to the organization of passenger flow, and helpful to the arrangement of shuttles, especially in large transit junctions. The application of automatic fare collection (AFC) devices in urban rail transit systems helps to collect large amounts of historical data of completed journeys, which can be used by metro operators to construct a database of the urban rail passenger volume. Based on deep learning techniques and big data, this paper develops an improved spatiotemporal long short-term memory model (Sp-LSTM) to forecast short-term outbound passenger volume at urban rail stations. The proposed model predicts the outbound passenger volume on the basis of the historical data of the spatial-temporal passenger volume, station origin–destination (OD) matrix and the operation data of the rail transit network. Finally, based on actual data of the Beijing Metro Airport Line, a case study is carried out to compare the proposed Sp-LSTM with other prediction methods, i.e., the general long short-term memory model (LSTM), the autoregressive integrated moving average model (ARIMA), and the non-linear autoregressive neural network model (NAR), and the results show that the proposed method outperforms the others.

Application of a new HMW framework derived ANN model for optimization of aquatic dissolved organic matter removal by coagulation

Removing dissolved organic matter (DOM) with polyaluminium chloride is one of the primary goals of drinking water treatment. In this study, a new HMW framework was proposed, which divided the factors affecting coagulation into three parts consisting of hydraulic condition (H), metal salt (M), and background water matrix (W). In this framework, H, M and W were assumed to be interacted with each other and combined to determine coagulation efficiency. We investigated the feasibility of the framework to determine the treatment efficiency through mathematical models. Results showed that non-linear artificial neural network (ANN) model was a better fit to the experimental data than the linear partial least squares (PLS) model: the ANN model could explain 76% of the total variations while the PLS could only explain 71%. The PLS did not follow the variations of observed values adequately. These experiments showed that the interaction between the HMW framework components were not simple linear relationships. The ANN model was able to optimize the composition of the HMW framework improving the efficiency of DOM removal through the components of HMW such as velocity gradient (G value), coagulant dosage, solution pH, and background water matrix. Overall, HMW framework is a new classification of factors affecting coagulation, leading to a better understanding of the coagulation process and sensitivity to influencing variables.

Scalable Planning with Deep Neural Network Learned Transition Models

In many complex planning problems with factored, continuous state and action spaces such as Reservoir Control, Heating Ventilation and Air Conditioning (HVAC), and Navigation domains, it is difficult to obtain a model of the complex nonlinear dynamics that govern state evolution. However, the ubiquity of modern sensors allows us to collect large quantities of data from each of these complex systems and build accurate, nonlinear deep neural network models of their state transitions. But there remains one major problem for the task of control – how can we plan with deep network learned transition models without resorting to Monte Carlo Tree Search and other black-box transition model techniques that ignore model structure and do not easily extend to continuous domains? In this paper, we introduce two types of planning methods that can leverage deep neural network learned transition models: Hybrid Deep MILP Planner (HD-MILP-Plan) and Tensorflow Planner (TF-Plan). In HD-MILP-Plan, we make the critical observation that the Rectified Linear Unit (ReLU) transfer function for deep networks not only allows faster convergence of model learning, but also permits a direct compilation of the deep network transition model to a Mixed-Integer Linear Program (MILP) encoding. Further, we identify deep network specific optimizations for HD-MILP-Plan that improve performance over a base encoding and show that we can plan optimally with respect to the learned deep networks. In TF-Plan, we take advantage of the efficiency of auto-differentiation tools and GPU-based computation where we encode a subclass of purely continuous planning problems as Recurrent Neural Networks and directly optimize the actions through backpropagation. We compare both planners and show that TF-Plan is able to approximate the optimal plans found by HD-MILP-Plan in less computation time. Hence this article offers two novel planners for continuous state and action domains with learned deep neural net transition models: one optimal method (HD-MILP-Plan) and a scalable alternative for large-scale problems (TF-Plan).

A comparative study on the prediction of the BP artificial neural network model and the ARIMA model in the incidence of AIDS

BackgroundAs a kind of widely distributed disease in China, acquired immune deficiency syndrome (AIDS) has been quickly growing each year, become a serious problem and caused serious damage to the life and health of people and the social events of China and the world because of its high fatality rate. It has been much concerned by all aspects of society. Therefore, developing early warning technology and finding the trend of early development are of quite significance to prevent and control human immunodeficiency virus (HIV)/AIDS. This study aimed to explore a suitable model for the morbidity of AIDS in China and establish a professional and feasible disease prediction model for the prevention and control works of AIDS.MethodsAt present, the traditional linear model is still utilized by most scholars to predict the incidence of HIV/AIDS. In addition, some scholars may attempt to use the nonlinear prediction model. Both prediction models showed good fitting and prediction effects. In China, the incidence of AIDS presents linear and nonlinear characteristics. In this research, the nonlinear back propagation artificial neural network (BP-ANN) model and the typical auto-regressive integrated moving average (ARIMA) linear model were applied to predict the incidence of HIV/AIDS and compare their fitting effects.ResultsBoth models were capable of predicting the expected cases of AIDS. It was seen that ARIMA and BP-ANN models could be used to forecast the monthly incidence of HIV/AIDS, but the fitting and forecasting effects of the nonlinear BP neural network model were better than those of the traditional linear ARIMA model.ConclusionsIn summary, it was further concluded that the BP-ANN model was a suitable way to monitor and predict the change trend and morbidity of AIDS in China.

Prediction of the droplet spreading dynamics on a solid substrate at irregular sampling intervals: Nonlinear Auto-Regressive eXogenous Artificial Neural Network approach (NARX-ANN)

This paper introduces a Nonlinear Auto-Regressive eXogenous Artificial Neural Network (NARX-ANN) model for prediction of spreading dynamics. 1220 experimental data of spreading dynamics of different droplets on various substrates have been collected from literature for model development. The model input parameters are Weber number, Ohnesorge number, and the tangent of equilibrium contact angle. An auxiliary input has been also added to take into account the irregular time sampling. D-optimal design of experiments has been utilized to determine the best parameters for the NARX model. It was found that the prediction capability of NARX model was better in comparison with KC and AGM models. The statistical parameters of the best developed NARX model including mean square error MSE=3.257×10-4, average absolute relative deviation (AARD) = 4.240%, and coefficient of determination (R2)=0.993 demonstrate the excellent prediction capability of the proposed model.

Design of a nonlinearly activated gradient-based neural network and its application to matrix inversion

Different from the traditional linearly activated gradient-based neural network model (GNN model), two nonlinear activation functions are presented and investigated to construct two nonlinear gradient-based neural network models (NGNN-1 model and NGNN-2 model) for matrix inversion in this paper. For comparative and illustrative purposes, the traditional GNN model is also used to solve matrix inversion problems under the same circumstance. In addition, the simulation results of the computer finally confirm the validity and superiority of the two nonlinear gradient-based neural network models specially activated by two nonlinear activation functions for matrix inversion, as compared with the traditional GNN model.

Combined application of electronic nose analysis and back-propagation neural network and random forest models for assessing yogurt flavor acceptability

Flavor acceptability is an important aspect of evaluating the quality of food products. Rapid flavor measurements and the detection of unsatisfactory products are thus necessary for the quality control of yogurt. To better evaluate the flavor acceptability of yogurt, this study used a method based on an electronic nose and nonlinear chemometric back-propagation neural network (BPNN) and random forest (RF) models. Initially, principal component analysis was applied to visualize the quality distribution of a set of yogurt samples, but it failed to distinguish between the satisfactory and unsatisfactory samples. However, the BPNN and RF models clearly discriminated between the two sample types, with accuracy values close to 100%. The RF model achieved better discrimination than the BPNN model, with an accuracy of 93.75% for three subsets of the samples with unsatisfactory flavor. In summary, the combination of an electronic nose and a nonlinear chemometric model is an effective system for the assessment of yogurt flavor acceptability.

Forecasting Short-term Container Vessel Traffic Volume Using Hybrid ARIMA-NN Model

A combination of linear and non-linear models results in a more accurate prediction in comparison with using linear or non-linear models individually to forecast time series data. This paper utilizes the linear autoregressive integrated moving average (ARIMA) model and non-linear artificial neural network (ANN) model to develop a new hybrid ARIMA-ANN model for prediction of container vessel traffic volume. The suggested hybrid method consists of an optimized feed-forward, back-propagation model with a hybrid training algorithm. The database of monthly traffic of Rajaee Port for thirteen years from 2005-2018 is taken into account. The performance of the developed model in forecasting short-term traffic volume is evaluated using various performance criteria such as correlation coefficient (R), mean absolute deviation (MAD), mean squared error (MSE) and mean absolute percentage error (MAPE). The developed model provides useful insights into container traffic behavior. Comparing the results with the real data-sets demonstrates the superior performance of the hybrid models than using models individually in forecasting traffic data.

Implementation of an efficient extreme learning machine for node localization in unmanned aerial vehicle assisted wireless sensor networks

SummaryAccurate node localization in wireless sensor networks (WSNs) is an essential for many networking protocols like clustering, routing, and network map building. The classical localization techniques such as multilateration and optimization‐based least square localization (OLSL) techniques estimate position of unknown node (UN) from the distance measured between all anchor nodes (ANs) and UNs. On the other hand, node localization using fixed terrestrial ANs suffers from poor localization accuracy because the ground to ground (GG) channel link is not reliable. By contrast, the mobile anchor deployed in unmanned aerial vehicle (UAV) provides high localization accuracy through reliable air to ground (AG) channel link. Still, the nonlinear distortion introduced in the wireless channel makes the distance measurement noisy. This noisy distance measurement also limits localization accuracy of classical localization techniques. Hence, the highly nonlinear artificial neural network (ANN) models such as multilayer perceptron (MLP) models can be applied effectively for node localization in UAV‐assisted WSNs. However, the MLP suffers from slow training speed, which limits its usage in real‐time applications. So, the extreme learning machine (ELM) is found to be a better alternative because it works on empirical error minimization theory, and its learning process requires only a single iteration. The detailed simulation analysis supports the proposed ELM localization scheme in terms of both localization accuracy and computational complexity.

Optimized nonlinear neural network architectural models for multistep wind speed forecasting

There is a growing demand for power from day to day. At present, the development achieved in power production from wind is highly significant. In this work, an optimized nonlinear neural network architectural model integrated with a modified firefly algorithm and particle swarm optimization is proposed to perform multistep wind-speed forecasting for specific target sites. Considering these aspects, this paper intends to predict wind speed, as its influence is high in generating wind power. The weights and bias values of the nonlinear neural network model are optimized employing the proposed optimization algorithm in order to achieve the minimum-error criterion. The computed results establish the effectiveness of the forecasting accuracy and the minimal error rate in comparison with existing methods available in the literature.

Learning-based Parameter Estimation for Hysteresis Modeling in Robotic Catheterization.

In the last half decade, nearly 31% of annual global deaths are linked to cardiovascular diseases. Thus, robotic catheterizations are recently proposed for interventions of conditions such as aneurism or atherosclerosis formed along vascular paths leading to the heart. However, existence of mild to strong hysteresis while navigating unactuated catheters with the current robotic systems inhibits autonomous control for vascular surgery. Thus, immersion of surgeons remains high with most of their time spent on steering the catheter in-and- out of the vessels. In this study, an autoregressive nonlinear neural network model is adapted for parameterization of vital causal factors of hysteresis during robotic catheterization. Crucial for autonomous control, hysteretic behaviors of endovascular tool are modeled while suitable values are estimated and analyzed for five contributory factors. The network model is validated with hysteresis data we obtained from a two degree-of-freedom robotic system and an unactuated catheter. Result validation shows accurate description of the hysteresis profile recorded during catheterization trials with a vascular phantom model.

Adaptive Market Hypothesis and Artificial Neural Networks: Evidence from Pakistan

The debate covering stock return predictability is now shifted towards the investigation of changing patterns of return predictability as suggested by the adaptive market hypothesis (AMH). The present article inspects the varying return predictability pertaining to the equity market in Pakistan under AMH framework. A nonlinear autoregressive neural network (NARNN) model is employed to investigate the nonlinear dependency of returns over a period of eighteen years. NARNN is a robust and flexible technique that is free from any restrictive assumptions. Under a rolling window framework, the repeating patterns of predictability and unpredictability are observed. This finding confirms the idea of AMH.

A Lithium-ion Battery RUL Prediction Method Considering the Capacity Regeneration Phenomenon

Prediction of Remaining Useful Life (RUL) of lithium-ion batteries plays a significant role in battery health management. Battery capacity is often chosen as the Health Indicator (HI) in research on lithium-ion battery RUL prediction. In the rest time of batteries, capacity will produce a certain degree of regeneration phenomenon, which exists in the use of each battery. Therefore, considering the capacity regeneration phenomenon in RUL prediction of lithium-ion batteries is helpful to improve the prediction performance of the model. In this paper, a novel method fusing the wavelet decomposition technology (WDT) and the Nonlinear Auto Regressive neural network (NARNN) model for predicting the RUL of a lithium-ion battery is proposed. Firstly, the multi-scale WDT is used to separate the global degradation and local regeneration of a battery capacity series. Then, the RUL prediction framework based on the NARNN model is constructed for the extracted global degradation and local regeneration. Finally, the two parts of the prediction results are combined to obtain the final RUL prediction result. Experiments show that the proposed method can not only effectively capture the capacity regeneration phenomenon, but also has high prediction accuracy and is less affected by different prediction starting points.

SoC Estimation for Lithium-Ion Battery Using Recurrent NARX Neural Network and Genetic Algorithm

State of charge (SOC) is an important indicator for assessing the remaining capacity of the battery. An accurate SOC estimation is crucial for ensuring the safe operation of lithium batteries and preventing from over-charging or over-discharging in electric vehicle (EV) industry. However, to estimate an accurate capacity of SOC of the lithium batteries has become a major concern for the EV industry. In this paper, a recurrent nonlinear autoregressive external input neural network(NARXNN) model optimized by genetic algorithm(GA) is proposed to improve accuracy of SOC of lithium battery by finding the optimal value of input delays, feedback delays, and hidden layer neurons. The NARXNN based GA model is compared with the NARXNN in performance using statistical error values of mean absolute error and root mean square error are used to check the performance of the SOC estimation. The results show that the NARXNN based genetic algorithm outperforms NARXNN in estimating SOC with high accuracy.

A regime-switching recurrent neural network model applied to wind time series

This paper proposes a regime-switching recurrent network model (RS-RNN) for non-stationary time series. The RS-RNN model emits a mixture density with dynamic nonlinear regimes that fit flexibly data distributions with non-Gaussian shapes. The key novelties are: development of an original representation of the means of the component distributions by dynamic nonlinear recurrent networks, and derivation of a corresponding expectation maximization (EM) training algorithm for finding the model parameters. The results show that the RS-RNN applied to a real-world wind speed time series achieves standardized residuals similar to popular previous models, but it is more accurate distribution forecasting than other linear switching (MS-AR) and nonlinear neural network (MLP and RNN) models.

Neural Network Approach to Forecast Hourly Intense Rainfall Using GNSS Precipitable Water Vapor and Meteorological Sensors

This work presents a methodology for the short-term forecast of intense rainfall based on a neural network and the integration of Global Navigation and Positioning System (GNSS) and meteorological data. Precipitable water vapor (PWV) derived from GNSS is combined with surface pressure, surface temperature and relative humidity obtained continuously from a ground-based meteorological station. Five years of GNSS data from one station in Lisbon, Portugal, are processed. Data for precipitation forecast are also collected from the meteorological station. Spaceborne Spinning Enhanced Visible and Infrared Imager (SEVIRI) data of cloud top measurements are also gathered, providing collocated information on an hourly basis. In previous studies it was found that the time-varying PWV is correlated with rainfall and can be used to detected heavy rain. However, a significant number of false positives were found, meaning that the evolution of PWV does not contain enough information to infer future rain. In this work, a nonlinear autoregressive exogenous neural network model (NARX) is used to process the GNSS and meteorological data to forecast the hourly precipitation. The proposed methodology improves the detection of intense rainfall events and reduces the number of false positives, with a good classification score varying from 63% up to 72% and a false positive rate of 36% down to 21%, for the tested years in the dataset. A score of 64% for intense rain events classification with 22% false positive rate is obtained for the most recent years. The method also achieves an almost 100% hit rate for the rain vs no rain detection, with close to no false alarms.