Time Series Forecasting Research Articles

Functional time series forecasting: a systematic review

Functional time series forecasting: a systematic review

Nonstationary Functional Time Series Forecasting

ABSTRACTWe propose a nonstationary functional time series forecasting method with an application to age‐specific mortality rates observed over the years. The method begins by taking the first‐order differencing and estimates its long‐run covariance function. Through eigendecomposition, we obtain a set of estimated functional principal components and their associated scores for the differenced series. These components allow us to reconstruct the original functional data and compute the residuals. To model the temporal patterns in the residuals, we again perform dynamic functional principal component analysis and extract its estimated principal components and the associated scores for the residuals. As a byproduct, we introduce a geometrically decaying weighted approach to assign higher weights to the most recent data than those from the distant past. Using the Swedish age‐specific mortality rates from 1751 to 2022, we demonstrate that the weighted dynamic functional factor model can produce more accurate point and interval forecasts, particularly for male series exhibiting higher volatility.

Predictive Abilities of Machine Learning and Deep Learning Approaches for Exchange Rate Prediction

This study evaluates the efficacy of forecasting models in predicting USD/TRY exchange rate fluctuations. We assess Support Vector Machine (SVM), XGBoost, Long Short-Term Memory (LSTM), and Gated Recurrent Unit (GRU) models with 96 and 21 feature sets. Data from 01.01.2010 to 30.04.2024 were sourced from Bloomberg, CBRT, and BDDK. Findings indicate that LSTM and GRU models outperform traditional models, with GRU showing the highest predictive accuracy. SVM performs poorly with highdimensional data, while XGBoost offers moderate predictive power but lacks in capturing intricate patterns. This study highlights the importance of model and feature selection in financial time series forecasting and underscores the advantages of advanced neural networks. The results provide valuable insights for analysts and policymakers in developing robust economic forecasting models.

FPGA-based Block Minifloat Training Accelerator for a Time Series Prediction Network

Time series forecasting is the problem of predicting future data samples from historical information and recent deep neural networks (DNNs) based techniques have achieved excellent results compared with conventional statistical approaches. Many applications at the edge can utilise this technology and most implementations have focused on inference, an ability to train at the edge would enable the deep neural network (DNN) to adapt to changing conditions. Unfortunately, training requires approximately three times more memory and computation than inference. Moreover, edge applications are often constrained by energy efficiency. In this work, we implement a block minifloat (BM) training accelerator for a time series prediction network, N-BEATS. Our architecture involves a mixed precision GEMM accelerator that utilizes block minifloat (BM) arithmetic. We use a 4-bit DSP packing scheme to optimize the implementation further, achieving a throughput of 779 Gops. The resulting power efficiency is 42.4 Gops/W, 3.1x better than a graphics processing unit in a similar technology.

PROVING OPTIMAL MODEL SELECTION AND ZERO COEFFICIENT CASES IN TIME SERIES FORECASTING WITH THE GENERALIZED LEAST DEVIATION ALGORITHM

This article provides a comprehensive analysis and proof of the Generalized Least Deviation Method (GLDM) in the context of time series forecasting, with a particular focus on optimal model order selection and the conditions that lead to zero coefficients. Central to this study is the GLDM Estimator, which determines the coefficients {aj} n(m) j=1 by minimizing the objective function F(a), defined as the sum of the arctangents of the absolute deviations from observed time series data {yt} T t=1 ⊂ R. The research not only proves GLDM’s ability to capture complex data interactions but also demonstrates its adaptability to varying model orders, showing that the selection of the optimal model order is influenced by the underlying characteristics of the data rather than just the data size. For example, temperature data with pronounced seasonal patterns and autocorrelations demands a fifth-order model, whereas wind speed and COVID-19 death cases in Russia are effectively modeled by a second-order structure. The study further examines the implications of higher-order models, advocating for a customized approach to model selection that enhances both predictive accuracy and interpretability in time series forecasting.

Heterogeneous Graph Transformer Auto-Encoder for multivariate time series forecasting

Multivariate time series (MTS) forecasting is crucial for optimizing intelligent applications. However, it is challenging because it requires the consideration of variable dependencies and temporal patterns based on complex relationships. Existing methods usually assume a single relationship between different variates, which results in the model learning a prominent shared temporal pattern. To address this problem, we innovatively use the heterogeneous graph to model multivariate time series. Specifically, we design the Heterogeneous Graph Transformer Auto-Encoder (HGTAE) to address this issue. In the data preprocessing stage, we construct MTS into the heterogeneous graph based on a priori knowledge of multiple relationships among different variates. Then, we design an adaptive module in HGTAE to embed different types of variates into the same dimension, which can uniformly represent the node features in the heterogeneous graph for subsequent graph processing and analysis. To learn the relationships and temporal information in the heterogeneous graph, we design an auto-encoder module in HGTAE. It learns correlations between nodes and their neighbors based on meta-relations, capturing temporal dependencies to reconstruct node attributes. We conduct link prediction and node prediction experiments on real-world heterogeneous graph datasets, and the results demonstrate the effectiveness of HGTAE in MTS forecasting tasks.

Chain-structured neural architecture search for financial time series forecasting

AbstractNeural architecture search (NAS) emerged as a way to automatically optimize neural networks for a specific task and dataset. Despite an abundance of research on NAS for images and natural language applications, similar studies for time series data are lacking. Among NAS search spaces, chain-structured are the simplest and most applicable to small datasets like time series. We compare three popular NAS strategies on chain-structured search spaces: Bayesian optimization (specifically Tree-structured Parzen Estimator), the hyperband method, and reinforcement learning in the context of financial time series forecasting. These strategies were employed to optimize simple well-understood neural architectures like the MLP, 1D CNN, and RNN, with more complex temporal fusion transformers (TFT) and their own optimizers included for comparison. We find Bayesian optimization and the hyperband method performing best among the strategies, and RNN and 1D CNN best among the architectures, but all methods were very close to each other with a high variance due to the difficulty of working with financial datasets. We discuss our approach to overcome the variance and provide implementation recommendations for future users and researchers.

Stock Price Prediction System LSTM Based on Deep Learning

Stock price prediction is a highly complex task due to the inherent volatility and non-linear patterns of financial markets. Recent advancements in deep learning, particularly Long Short-Term Memory (LSTM) networks, have yielded promising results in time series forecasting. This paper provides an in-depth analysis of LSTM models applied to the stock price prediction of major technology companies, including Apple (AAPL), Tesla (TSLA), Microsoft (MSFT), and Nvidia (NVDA). The study utilizes historical stock data to train and evaluate LSTM models, comparing their performance against various other regression models such as Linear Regression, Support Vector Regression (SVR), Decision Tree Regression, and Gradient Boosting Decision Trees (GBDT).The results demonstrate that the LSTM model outperforms traditional models, particularly in its ability to capture long-term dependencies and manage complex, volatile stock data. Among the companies analyzed, the LSTM model achieved the most accurate predictions for Apple, as indicated by the lowest Mean Squared Error (MSE). Tesla's predictions, while less accurate due to high volatility, still showed improvement compared to other models. In conclusion, the LSTM network proved to be the most effective method for stock price prediction in this study, particularly excelling in forecasting long-term trends and reducing prediction errors for volatile stocks.

Time Series Modelling and Prediction of WTI Crude Oil Prices Trend using Machine Learning Techniques

It is challenging to forecast crude oil prices. Given that the international crude oil price is time-varying and nonlinear, suggest a new mixed method for oil price forecasting. First, the traditional time series technique ARIMA method for predictions is incorporated. Then, The time-varying and nonlinear components of crude oil prices are forecasted using machine learning approaches such as boosted trees, SVM, ANN, LSTM, bagged trees and linear regression, etc. The ultimate predicted outcomes of crude oil prices are then calculated by adding the predicted prices of each component. The results showed that the new proposed hybrid method was capable of a high degree of oil price prediction based on its exceptional ability to adapt sample selection, data frequencies and structural breaks. In addition, the comparisons show that the new method proves to be more accurate than these well-known methods for crude oil price forecasts. The WTI daily data from 1986 to 2018 has been utilised for forecasting of crude oil trend for 2018 in next 50 days. Forecasts were made using the time series forecasting approach with the ARIMA model and the machine learning approach. Validation metrics have been used to evaluate the performance of these models and it was noted that they performed well. The effects of prediction methods on crude oil data are discussed with the help of graphs for the predictions for next 50 days, actual and predicted plots, model performance plot, and validation performance plot for all the methods. Compared to the previous published work, the accuracy of the problem has been verified by comparing it to the quantitative outputs exhibited in this, and the agreement between the results is excellent, which has led to confidence in the quantitative results projected in this study.. KEYWORDS :Crude oil price, ARIMA model, Time series forecasting, Machine learning techniques, Error metrics.

Forecasting Strategies in Retail: Utilizing Advanced Machine Learning Methods while Safeguarding Privacy

Abstract Demand forecasting is critical to optimizing retail inventory management, pricing, and operations. This study compares several time series forecasting models on a publicly available retail dataset, including traditional stats like ARIMA and SARIMA, smoothing techniques, and new ones from Facebook, such as Prophet and Neural Prophet. We evaluate each model using metrics like Mean Absolute Error (MAE), Root Mean Squared Error (RMSE), and R2. The results show that NeuralProphet and hybrid models outperform traditional models like ARIMA and SARIMA. We also look at the computational efficiency and practicality of the models and provide insights into how they can be used in real-world demand forecasting scenarios. The results show that traditional models are robust. Still, advanced models like NeuralProphet and hybrids, which combine machine learning with time series forecasting, have a lot of potential to improve forecast accuracy and operational decision-making.

A Comparative Analysis of Automated Machine Learning Libraries for Electricity Price Forecasting

Abstract Reliable and accurate electricity price forecasting algorithms can be used to inform efficient energy consumption schedules and maximise profits for electricity traders. Operating within Ireland’s Integrated Single Electricity Market (I-SEM), traders can buy and sell electricity at fluctuating hourly rates whose day-ahead prices are published at approximately 13:00 GMT day-1. Access to electricity price predictions earlier than this publication time allows stakeholders an expanded timeframe to facilitate energy cost-aware scheduling. While many studies have been conducted to espouse various machine learning and statistical approaches to electricity price forecasting, these models tend to be bespoke and require in-depth knowledge regarding model implementation. The problem of requiring such expertise is not unique to time series forecasting, and research into mitigating such limitations exists in the form of Automated Machine Learning (AutoML). AutoML aims to derive effective models while automating various steps typically required for machine learning experimentation, such as pre-processing, model selection, validation, etc. Given the increasing proliferation of AutoML tools and frameworks, this paper applies eight Python-based AutoML libraries to day-ahead electricity price forecasting on an excerpt of I-SEM data. These libraries are compared across a series of error metrics and training times to produce an empirical benchmark that can be utilised to select high-performing AutoML tools for further price forecasting research and other forms of time series forecasting. AutoKeras is found to produce accurate forecasts but requires careful configuration to avoid long runtimes. PyCaret, Ludwig, FLAML and FEDOT also generate favourable results while being significantly easier to configure.

Непараметрическая оценка прогностической точности моделей на примере временны́х рядов концентраций метана в атмосферном воздухе арк­тического острова Белый

Over the past few decades, the number, variety, and complexity of time series forecasting models have grown. There has also been an increased interest among researchers in comparing and assessing the predictive accuracy and performance of models, and in determining which models are more accurate. The researchers propose using the hypothesis testing approach to assess the performance of a time series forecast model. They obtained data for the study while monitoring the dynamics of ground-level concentrations of the main greenhouse gases on the Arctic Island of Bely, Yamalo-Nenets Autonomous Area, Russia. A total of three models based on autoregressive neural networks with exogenous input (NARX) were considered to predict changes in methane concentration in the surface layer of atmospheric air. The performance of the models was assessed using 8 “traditional” indices (correlation and determination coefficients, Wilmott goodness-of-fit indices, mean absolute error, relative mean square error, etc.) and the proposed permutational approach using the hypothesis testing method. In general, based on the data under study, the estimates of the permutation approach and other indicators of accuracy and error coincide, but there are discrepancies. Effect sizes do not always determine the statistical significance of differences.

Forecasting the Metal Ores Industry Index on the Tehran Stock Exchange: A Gated Recurrent Unit (GRU) Approach

This research offers an in-depth examination of predicting the closing prices of the metal ores industry index on the Tehran Stock Exchange (TSE) using a Gated Recurrent Unit (GRU) model. The GRU, a type of recurrent neural network, shows great promise for tasks involving time series forecasting. The historical daily price data from October 2017 to October 2022, was used in the study after carefully preprocessing it for further analysis. The study begins with a univariate analysis to reveal distribution characteristics and the relationships between essential variables. A customized GRU model that is trained on 70% of the time series data, with its performance assessed through metrics such as Root Mean Square Error (RMSE), Mean Square Error (MSE), Mean Absolute Error (MAE), and the R-squared (R2) score is used for prediction. The results indicate that the GRU model provides accurate predictions for the metal ores industry index, outperforming traditional forecasting techniques. The model's recurrent nature enables it to capture both short-term and long-term temporal dependencies within the data. This research highlights the significant potential of GRU networks in the realm of financial forecasting. Future improvements will focus on hyperparameter optimization and further integrating additional input variables to enhance predictive accuracy.

Stock Market Price Prediction Using ARIMA Model

Stock market price prediction involves forecasting future stock values using techniques like technical analysis, fundamental analysis, and machine learning. Technical analysis looks at historical price movements, while fundamental analysis evaluates a company's financial health. Machine learning models use large datasets to identify patterns. This paper introduces a vital approach to time series analysis and forecasting, emphasizing its widespread practical applications. Time series data consists of an ordered sequence of data points recorded at equal time intervals. The stock market, one of the most complex financial systems, experiences significant fluctuations in stock prices over time. Stock market forecasting aims to uncover market trends and help investors maximize returns while minimizing risks. Given the market's sensitivity to rapid changes, this research focuses on developing innovative approaches to forecast stocks for high profits. Specifically, this paper analyzes the time-series data of the Indian stock market and builds a statistical model to predict future stock prices efficiently

An Improved Parallel Heterogeneous Long Short-Term Memory Model with Bayesian Optimization for Time Series Prediction

Currently, Deep Learning (DL) with the Recurrent Neural Networks (RNN) variants is being applied successfully in many domains of Engineering for prediction. In view of the demand for precise forecasting and the aid of Artificial Intelligence Tools, time series prediction reveals a vital task in decision-making and risk assessment. However, the application of novel Recurrent DL models for obtaining an accurate prediction of time series is yet to be explored. Recent trends reveal that Hybrid Neural Networks and DL models are appropriate for time series forecasts. At the same time, the model's selection and the hyperparameter's tuning can greatly impact its performance. To address this problem, a parallel long-term memory (PLSTM) model integrated with Bayesian hyperparameter optimization (PLSTM-BO) is proposed for time series prediction. The model is tuned in terms of key parameters, including the number of neurons, dropout, learning rate, and optimization technique. The model's performance is assessed using the SARS-COVID-19 cumulative cases, deaths, recovery cases, and NIFTY 50 stock closing price time series dataset. The obtained results convey that the current model exhibits remarkable performance compared to existing models.

Efficient Integration of CNN and LSTM for Data Mining Observation Algorithms: A Systematic Review

An extensive analysis of the combination of Long Short-Term Memory (LSTM) networks with Convolutional Neural Networks (CNN) is presented in this research. within data mining applications. CNNs, optimized for spatial data processing, and LSTMs, made to manage data in a sequential fashion, are combined to leverage their respective strengths in feature extraction and temporal pattern recognition. By analyzing over 60 published studies, we evaluate the effectiveness of CNN-LSTM models across diverse domains such as medical image analysis, financial time series forecasting, stock price prediction, and sentiment analysis. The review reveals that CNN-LSTM hybrids significantly outperform traditional methods in activities requiring both temporally and spatially understanding. However, challenges such as model interpretability, hyperparameter optimization, and the need for diverse datasets remain. Future research should focus on optimizing these models for specific applications, improving scalability, and addressing limitations like data scarcity. This review underscores the significant potential of CNNLSTM integration in advancing data mining techniques and solving complex problems in various fields. We present the related work of some of the most important publication in the field and classify over 60 published papers from different perspective. This review intends to classify the major techniques, methods, algorithms in the domain of medical images, financial time series analysis, stock price and social media.

Enhancing financial portfolio management with predictive analytics and scalable data modeling techniques

The financial industry faces an increasingly complex landscape for portfolio management, where data-driven insights are crucial for optimizing asset allocation and managing risk. This paper explores the integration of predictive analytics and scalable data modeling techniques in enhancing financial portfolio management. By leveraging machine learning algorithms, big data architectures, and real-time data processing, predictive analytics can forecast asset trends, detect market anomalies, and assess portfolio risk with high precision. We evaluate several predictive models, including time-series forecasting, neural networks, and ensemble methods, for their efficacy in financial prediction. The study also discusses the role of scalable data modeling frameworks, such as Apache Spark and cloud-based data lakes, in handling vast volumes of unstructured data across different markets and asset classes. Findings indicate that predictive analytics, when paired with robust data models, can deliver real-time, actionable insights, enhancing decision-making for fund managers and institutional investors. Furthermore, this paper highlights best practices for implementing scalable models in financial institutions, addressing challenges like data latency, model interpretability, and system scalability. By adopting these advanced analytics frameworks, portfolio managers can achieve improved risk-adjusted returns, better asset diversification, and enhanced adaptability to market volatility.

TIME SERIES, MACHINE LEARNING AND ENSEMBLE MODELS TO FORECAST RICE CROP YIELD IN NIGERIA UNDER CLIMATIC VARIABILITY

The rising demand for food globally due to unprecedented population growth has led to food insecurity in some populated regions, such as Africa. This leads to global agricultural production to be of increasing concern. The growing requirement for food in an era of a changing climate and scarce resources has inspired substantial crop yield prediction research. Another contributing factor to global food insecurity is climate change and its variability. The impact of increasing climate variability on rice crop yield is now evident. Predicting the potential effects of climate change on rice crops prompts the use of time series, machine learning, and ensemble models. The ensemble model was designed by assigning weights to the individual models based on their performance metrics, such as R-squared values and error rates. Results showed that the ensemble model achieved superior accuracy, with an R-squared value of 0.9523, surpassing the performance of the individual SARIMAX (Seasonal AutoRegressive Integrated Moving Average with eXogenous Regressors) and LSTM (Long Short-Term Memory)models. The predicted yields aligned closely with observed values in both training and testing phases, demonstrating the model's ability to capture historical trends and generalise effectively to unseen data. This study shows the importance of integrating time-series forecasting and machine learning techniques to improve rice crop yield predictions.

Intelligent hybrid approaches utilizing time series forecasting error for enhanced structural health monitoring

Over the past decade, the growing importance of machine learning-based structural health monitoring (SHM) for early-stage damage detection has become evident. Time series forecasting, using deep learning, has emerged as a key focus, significantly contributing to improving damage detection, localization, and quantification processes. Researchers in SHM have conducted numerous studies utilizing neural networks based on time series forecasting, grounded in traditional methods. This study diverges from existing research by directly incorporating neural network prediction errors in time series for detecting, localizing, and quantifying damage. The proposed methods are well-suited for online structural monitoring. They eliminate the need for data classification methods and damage-sensitive feature extraction techniques by relying solely on training the neural network with data from structurally sound conditions. However, the testing process does require data from damaged conditions. To address the non-linear and non-stationary characteristics of the signals, the Improved Complete Ensemble Empirical Mode Decomposition with Adaptive Noise (ICEEMDAN) method is applied for signal processing. This method processes response signals (i.e., time series) from three well-known benchmark structures: the University of Central Florida structure, the Qatar University Grandstand Simulator, and the Z24 Bridge. Subsequently, the first intrinsic mode function (IMF) obtained from signal decomposition is independently input into Long-Short Term Memory (LSTM) and Gated Recurrent Unit (GRU) neural networks for time series prediction. Optimal parameter values for the LSTM and GRU neural networks are chosen using the Bayesian Optimization (BO) algorithm before the prediction process. By introducing three indices—Statistical Distance Function (SDF), error index, and accuracy index—the evaluation not only emphasizes the accuracy of the methods but also explores the localization and quantification of damage. The results demonstrate that both ICEEMDAN-BO-LSTM-SDF and ICEEMDAN-BO-GRU-SDF methods have successfully achieved accurate detection, localization, and quantification without the need for data classification and damage-sensitive feature extraction methods, and merely by utilizing data from healthy states for neural network training.

Probabilistic Time Series Forecasting Based on Similar Segment Importance in the Process Industry

Probabilistic time series forecasting is crucial in various fields, including reducing stockout risks in retail, balancing road network loads, and optimizing power distribution systems. Building forecasting models for large-scale time series is challenging due to distribution differences, amplitude fluctuations, and complex patterns across various series. To address these challenges, a probabilistic forecasting method with two different implementations that focus on historical segment importance is proposed in this paper. First, a patch squeeze and excitation (PSE) module is designed to preprocess historical data, capture segment importance, and distill information. Next, an LSTM-based network is used to generate maximum likelihood estimations of distribution parameters or different quantiles for multi-step forecasting. Experimental results demonstrate that the proposed PSE module significantly enhances the base model’s prediction performance, and direct multi-step forecasting offers more detailed information for high-frequency data than recursive forecasting.