Networks For Time Series Forecasting Research Articles

Predicting Bitcoin and Ethereum prices using long short-term memory and gated recurrent unit

Predicting future prices of cryptocurrencies, including Bitcoin and Ethereum, presents a formidable challenge owing to their inherent volatility. This study applies Long Short-Term Memory (LSTM), a well-established recurrent neural network for time series forecasting, to predict Bitcoin and Ethereum values. Historical price data for both cryptocurrencies, sourced from Yahoo Finance, serves as the basis for analysis. The dataset undergoes an 80% training and 20% testing partition. Subsequently, LSTM models are developed and trained on both datasets. In parallel, the gated recurrent unit (GRU), recognized as an advanced variant of the LSTM model, is explored for comparative purposes. Performance evaluation utilizes fundamental metrics, including root mean squared error (RMSE), mean absolute error (MAE), and mean absolute percentage error (MAPE). The results reveal an intriguing trend: both models exhibit superior performance when applied to the Ethereum dataset compared to the Bitcoin dataset. This observation suggests the potential presence of Ethereum-specific features or patterns that align more effectively with deep learning model architectures. Notably, the GRU model consistently outperforms the LSTM model across RMSE, MAE, and MAPE. These outcomes underscore the GRU model’s capacity as a robust tool for cryptocurrency value prediction. In summary, this study tackles the challenge of cryptocurrency price prediction while emphasizing the promising role of advanced neural network architectures, such as GRU, in enhancing prediction accuracy, thus offering valuable insights into financial forecasting.

Exploring the Use of Recurrent Neural Networks for Time Series Forecasting

Exploring the Use of Recurrent Neural Networks for Time Series Forecasting

Exploring the Use of Recurrent Neural Networks for Time Series Forecasting

Exploring the Use of Recurrent Neural Networks for Time Series Forecasting

A Combined Model Based on Recurrent Neural Networks and Graph Convolutional Networks for Financial Time Series Forecasting

Accurate and real-time forecasting of the price of oil plays an important role in the world economy. Research interest in forecasting this type of time series has increased considerably in recent decades, since, due to the characteristics of the time series, it was a complicated task with inaccurate results. Concretely, deep learning models such as Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs) have appeared in this field with promising results compared to traditional approaches. To improve the performance of existing networks in time series forecasting, in this work two types of neural networks are brought together, combining the characteristics of a Graph Convolutional Network (GCN) and a Bidirectional Long Short-Term Memory (BiLSTM) network. This is a novel evolution that improves existing results in the literature and provides new possibilities in the analysis of time series. The results confirm a better performance of the combined BiLSTM-GCN approach compared to the BiLSTM and GCN models separately, as well as to the traditional models, with a lower error in all the error metrics used: the Root Mean Squared Error (RMSE), the Mean Squared Error (MSE), the Mean Absolute Percentage Error (MAPE) and the R-squared (R2). These results represent a smaller difference between the result returned by the model and the real value and, therefore, a greater precision in the predictions of this model.

ВИКОРИСТАННЯ ІНТЕРНЕТ-ТЕХНОЛОГІЙ ДЛЯ ДІАГНОСТИКИ ТА ПРОГНОЗУВАННЯ ПРОЦЕСІВ ПРИЙНЯТТЯ БІЗНЕС-РІШЕНЬ

The article analyzes the application of fractal analysis and neural networks in internet technologies for diagnosing the state of a studied object and forecasting time series. These methods will expand the possibilities of using predictions for the future behavior of the researched object in the fields of finance, investment, economics, sociology, and other industries where obtaining prospective knowledge is required. The authors provide an overview of predictive analysis theories and modern econometric and mathematical methods for modeling time series. The justification for choosing fractal analysis as an effective method for diagnosing business decision-making processes is given. A methodology for using internet technology to automate time series forecasting is proposed. A web application has been developed as a software product to solve the research tasks. The process of developing the web application is examined, which would forecast the financial indicators of Ukrainian banks based on available time series. The main stages of development and architecture of the web application for diagnosing and forecasting business processes are described. A generalized structure of software modules is presented. Time series of net profit for Ukrainian banks have been selected as the base model for analysis and forecasting. The developed web application is a complex of programs that use a neural network for time series forecasting. Fractal analysis is used in the stage of forming input data for the neural network. The application also provides the ability to conduct R/S analysis of time series to determine the Hurst exponent. The software product also allows training the neural network using backpropagation. With the machine learning algorithms used in the web application, it is possible to analyze large volumes of data and discover complex dependencies and patterns in time series that may be difficult to notice manually. This leads to more accurate and faster results and enables making more objective forecasts based on the available data. The developed web application can also be used for predicting crises in financial and investment activities.

Recurrent Neural Networks for Time Series Forecasting. Choosing the best Architecture for Passenger Traffic Data

Accurately predicting the urban traffic passenger flow is of great importance for transportation resource scheduling, planning, public safety, and risk assessment. Traditional statistical approaches for forecasting time series are not effective in practice. They often require either strict or weak data stationarity, which is almost impossible to obtain with real data. An alternative method is time series forecasting using neural networks. By their nature, neural networks are non-linear and learn based on input and output data. With this approach, increasing the efficiency of the network is reduced to increasing the amount of data of the initial sample. Today, the class of recurrent neural networks is mainly used for forecasting time series. Another important stage is the choice of neural network architecture. In this article the use of long short term memory and gated recurrent units architecture is considered and also is compared their performance for passenger flow forecasting.

Joint training of a predictor network and a generative adversarial network for time series forecasting: A case study of bearing prognostics

The lack of bearing run-to-failure data has been one of the challenges in developing and practically implementing robust bearing prognostics models. This paper proposes a new Generative Adversarial Network (GAN) based prognostics method for RUL prediction. We propose a novel joint training strategy to integrate the training process of a bearing health predictor within the GAN architecture. GAN uses available time series degradation data to generate synthetic degradation data that enhances the predictor’s learning and forecast performance, thus improving the RUL prediction accuracy. We demonstrate the utility and performance of the proposed method through two examples. The first numerical toy case study of forecasting polynomial-like time series shows that the proposed Jointly Trained Health Predictor (HP-JT) method produces smaller one- and multi-step-ahead prediction errors than a traditional health predictor (HP). In the second case study, we design a cross-validation study utilizing an open-source bearing dataset to evaluate the model’s performance in RUL prediction. Compared to HP, the proposed method decreases the bearing RUL prediction average error by 29.4% in a five-fold cross-validation study. We further compare the model with standard data augmentation techniques such as adding noise and using a variational autoencoder (VAE). The results from the case studies show that the proposed method can generate time series representing the real-data distribution.

Spatial-Temporal Semantic Neural Network for Time Series Forecasting

Time series forecasting play an important role in many applications, and combining neural networks to forecast time series features can uncover many potential application situations. For example, forecasting traffic flow time series data is important for urban traffic planning and traffic management. Accurate forecasting of traffic flow provides the basis for road traffic control, which in turn improves traffic efficiency and reduces congestion. In this paper, STSeNN is proposed, which combines graph neural network and TCN to dynamically capture the spatial correlation, temporal correlation and semantic correlation of data. Experiments show that the prediction performance of STSeNN on the traffic flow datasets is more accurate compared with existing methods.

On the post-hoc explainability of deep echo state networks for time series forecasting, image and video classification

Since their inception, learning techniques under the reservoir computing paradigm have shown a great modeling capability for recurrent systems without the computing overheads required for other approaches, specially deep neural networks. Among them, different flavors of echo state networks have attracted many stares through time, mainly due to the simplicity and computational efficiency of their learning algorithm. However, these advantages do not compensate for the fact that echo state networks remain as black-box models whose decisions cannot be easily explained to the general audience. This issue is even more involved for multi-layered (also referred to as deep) echo state networks, whose more complex hierarchical structure hinders even further the explainability of their internals to users without expertise in machine learning or even computer science. This lack of explainability can jeopardize the widespread adoption of these models in certain domains where accountability and understandability of machine learning models is a must (e.g., medical diagnosis, social politics). This work addresses this issue by conducting an explainability study of echo state networks when applied to learning tasks with time series, image and video data. Among these tasks, we stress on the latter one (video classification) which, to the best of our knowledge, has never been tackled before with echo state networks in the related literature. Specifically, the study proposes three different techniques capable of eliciting understandable information about the knowledge grasped by these recurrent models, namely potential memory, temporal patterns and pixel absence effect. Potential memory addresses questions related to the effect of the reservoir size in the capability of the model to store temporal information, whereas temporal patterns unveil the recurrent relationships captured by the model over time. Finally, pixel absence effect attempts at evaluating the effect of the absence of a given pixel when the echo state network model is used for image and video classification. The benefits of the proposed suite of techniques are showcased over three different domains of applicability: time series modeling, image and, for the first time in the related literature, video classification. The obtained results reveal that the proposed techniques not only allow for an informed understanding of the way these models work, but also serve as diagnostic tools capable of detecting issues inherited from data (e.g., presence of hidden bias).

Evolving deep neural networks for Time Series Forecasting

The aim of this paper is to introduce a biologically inspired approach that can automatically generate Deep Neural networks with good prediction capacity, smaller error and large tolerance to noises. In order to do this, three biological paradigms were used: Genetic Algorithm (GA), Lindenmayer System and Neural Networks (DNNs). The final sections of the paper present some experiments aimed at investigating the possibilities of the method in the forecast the price of energy in the Brazilian market. The proposed model considers a multi-step ahead price prediction (12, 24, and 36 weeks ahead). The results for MLP and LSTM networks show a good ability to predict peaks and satisfactory accuracy according to error measures comparing with other methods.

Biosensing human blood clotting factor by dual probes: Evaluation by deep long short-term memory networks in time series forecasting.

Artificial intelligence of things (AIoT) has become a potential tool for use in a wide range of fields, and its use is expanding in interdisciplinary sciences. On the other hand, in a clinical scenario, human blood-clotting disease (Royal disease) detection has been considered an urgent issue that has to be solved. This study uses AIoT with deep long short-term memory networks for biosensing application and analyzes the potent clinical target, human blood clotting factor IX, by its aptamer/antibody as the probe on the microscaled fingers and gaps of the interdigitated electrode. The earlier results by the current-volt measurements have shown the changes in the surface modification. The limit of detection (LOD) was noticed as 1 pM with the antibody as the probe, whereas the aptamer behaved better with the LOD at 100 fM. The time-series predictions from the AIoT application supported the obtained results with the laboratory analyses using both probes. This application clearly supports the results obtained from the interdigitated electrode sensor as aptamer to be the better option for analyzing the blood clotting defects. The current study supports a great implementation of AIoT in sensing application and can be followed for other clinical biomarkers.

Recurrent Neural Networks for Time Series Forecasting: Current status and future directions

Recurrent Neural Networks (RNNs) have become competitive forecasting methods, as most notably shown in the winning method of the recent M4 competition. However, established statistical models such as exponential smoothing (ETS) and the autoregressive integrated moving average (ARIMA) gain their popularity not only from their high accuracy, but also because they are suitable for non-expert users in that they are robust, efficient, and automatic. In these areas, RNNs have still a long way to go. We present an extensive empirical study and an open-source software framework of existing RNN architectures for forecasting, and we develop guidelines and best practices for their use. For example, we conclude that RNNs are capable of modelling seasonality directly if the series in the dataset possess homogeneous seasonal patterns; otherwise, we recommend a deseasonalisation step. Comparisons against ETS and ARIMA demonstrate that (semi-) automatic RNN models are not silver bullets, but they are nevertheless competitive alternatives in many situations.

Forecasting Solar Cycle 25 Using Deep Neural Networks

With recent advances in the field of machine learning, the use of deep neural networks for time series forecasting has become more prevalent. The quasi-periodic nature of the solar cycle makes it a good candidate for applying time series forecasting methods. We employ a combination of WaveNet and LSTM neural networks to forecast the sunspot number using the years 1749 to 2019 and total sunspot area using the years 1874 to 2019 time series data for the upcoming Solar Cycle 25. Three other models involving the use of LSTMs and 1D ConvNets are also compared with our best model. Our analysis shows that the WaveNet and LSTM model is able to better capture the overall trend and learn the inherent long and short term dependencies in time series data. Using this method we forecast 11 years of monthly averaged data for Solar Cycle 25. Our forecasts show that the upcoming Solar Cycle 25 will have a maximum sunspot number around 106 $\pm$ 19.75 and maximum total sunspot area around 1771 $\pm$ 381.17. This indicates that the cycle would be slightly weaker than Solar Cycle 24.

Simulation Study for Determining the Best Architecture of Multilayer Perceptron for Forecasting Nonlinear Seasonal Time Series

Neural network is one of flexible nonlinear models that could handle various relationship patterns on data with high accuracy. The data-driven approach is one of the advantages of neural network models in solving complex problems in forecasting. The selection of the best model becomes one of the most important problems in the application of neural network for time series forecasting, which consists of determining the input, the number of neurons in the hidden layer, the activation function, and preprocessing method. This paper focuses on the simulation study to explore how to determine the best architecture of multilayer perceptron for forecasting nonlinear seasonal time series. The data that be generated from seasonal exponential smooth transition autoregressive model are used as a case study. The results show that the inputs and the number of neurons in the hidden layer are two main factors that affect significantly the forecast accuracy. In contrary, the activation function and preprocessing method do not significantly influence the forecast accuracy.

Optimal Forecast Combination Based on Neural Networks for Time Series Forecasting

Research indicates that forecast combination is one of the most important and effective approaches for time series forecasting. The success of forecast combination depends on how well component models are selected and combination weights are determined. A forecast combination model resulting from a new neural networks-based linear ensemble framework (NNsLEF) is proposed in this study. The principle of the proposed framework adheres to three primary aspects. (a) Four kinds of neural network models, namely, back-propagation neural network, dynamic architecture for artificial neural network, Elman artificial neural network, and echo state network, are selected as component forecasting models. (b) An input-hidden selection heuristic (IHSH) is designed to determine the input-hidden neuron combination for each component neural network. (c) An in-sample training–validation pair-based neural network weighting (ITVPNNW) mechanism is studied to generate the associated combination weights. In particular, the four neural network models are applied to impart their superior performance to the combination approach while maintaining their diversity. Meanwhile, IHSH is investigated to improve the performance of each component neural network model by attempting to solve the familiar overfitting problem of networks. Lastly, the ITVPNNW mechanism is studied to search for a set of appropriate combination weights that will primarily affect the accuracy of the linear ensemble framework. Resultsfrom experiments performed on eight time series data sets show that NNsLEF outperforms the four component neural network models and other well-recognized models.

A Feedback-oriented Data Delay Modeling in a Dynamic Neural Network for Time Series Forecasting

In this study, we develop a neural network with a time shifting approach to forecast time series patterns. We investigate the impact of different layer-weight configurations to capture the trends in the forms of seasonal, chaotic, etc. We also hypothesize the combined effect of the delayed inputs and the forward connections to introduce a dynamical structure. The effect of overfitting issue is procedurally monitored to gain the resistance property from the early stoppage of training process and to reduce the predictions' error. Finally, the performance of the proposed network is challenged by six well-known deterministic and non-deterministic time series and compared by the autoregression (AR), artificial neural network (ANN), adaptive k-nearest neighbors (AKN), and adaptive neural network (ADNN) models. The results show that the proposed network outperforms the conventional models, particularly in forecasting the chaotic and seasonal time series.

HYBRID MODULAR NEURAL NETWORKS

In this paper considered hybrid neural networks for time series forecasting and methods based on fuzzy models, such as ANFIS models and we also suggest a hybrid neural network model based on modular neural networks.

Methodological Advances in Artificial Neural Networks for Time Series Forecasting

Objective: The aim of this paper is to analyze the development of new forecasting models based on neural networks. Method: We used the systematic literature review method employing a manual search of papers published on new neural networks models in the time period 2000 to 2010. Results: Only 18 studies meet all the requirements of the inclusion criteria. Of these, only three proposals considered a neural networks model using a process different to the autoregressive. Conclusion: Although studies relating to the application of neural network models were frequently present, we find that the studies proposing new forecasting models based on neural networks with a theoretical support and a systematic procedure for the construction of model, were scarce in the time period 2000-2010.

A New Hybrid Grey Neural Network Based on Grey Verhulst Model and BP Neural Network for Time Series Forecasting

The advantages and disadvantages of BP neural network and grey Verhulst model for time series prediction are analyzed respectively, this article proposes a new time series forecasting model for the time series growth in S-type or growth being saturated. From the data fitting's viewpoint, the new model named grey Verhulst neural network is established based on grey Verhulst model and BP neural network. Firstly, the Verhulst model is mapped to a BP neural network, the corresponding relationships between grey Verhulst model parameters and BP network weights is established. Then, the BP neural network is trained by means of BP algorithm, when the BP network convergences, the optimized weights can be extracted, and the optimized grey Verhulst neural network model can be obtained. The experiment results show that the new model is effective with the advantages of high precision, less samples required and simple calculation, which makes full use of the similarities and complementarities between grey system model and BP neural network to settle the disadvantage of applying grey model and neural network separately. It is concluded that grey Verhulst neural network is a feasible and effective modeling method for the time series increasing in the curve with S-type.

Stacked Heterogeneous Neural Networks for Time Series Forecasting

A hybrid model for time series forecasting is proposed. It is a stacked neural network, containing one normal multilayer perceptron with bipolar sigmoid activation functions, and the other with an exponential activation function in the output layer. As shown by the case studies, the proposed stacked hybrid neural model performs well on a variety of benchmark time series. The combination of weights of the two stack components that leads to optimal performance is also studied.