Conventional Time Series Models Research Articles

A parallel and multi-scale probabilistic temporal convolutional neural networks for forecasting the key monitoring parameters of gas turbine

As a crucial equipment in the power industry, gas turbines need effective condition monitoring techniques to maintain safe and reliable operations. Fast yet accurate long-term forecasting of the key monitoring parameters of gas turbine is vital for achieving the overall effective condition monitoring. This however poses challenges in terms of both efficacy and efficiency for conventional time series models like recurrent neural networks (RNN), since they run in a sequential manner. To this end, this paper introduces a novel parallel probabilistic time series prediction model. Particularly, the proposed model leverages the parallelism of temporal convolutional neural network (TCN) and exploits the power of latent space in the Bayesian framework. Besides, a multi-scale feature extraction strategy based on dense connections and a lagged observation strategy are presented to improve the model performance. In the comparative study against state-of-the art competitors on four classical system identification benchmarks, the proposed model significantly reduces the training time, while consistently achieving highly accurate predictions and providing appropriate uncertainty quantification. Thereafter, the proposed model is adopted to build a systematic workflow for the fast yet accurate long-term forecasting of the temperature of blade channel of gas turbine. The comprehensive results again highlight the superiority of the proposed model in terms of both the prediction quality and the training efficiency.

Handling Volatility and Nonlinearity in Wind Speed Data: A Comparative Analysis between ARIMA-GARCH and ARIMA-MLP

One of the notable features of wind speed is its volatility and nonlinearity. Thorough assessment on the presence of these features is crucial to obtain a wind speed forecasting model with higher accuracy. In this study, the conventional time series linear model; ARIMA model was applied to assess the internal structure of the wind speed daily data in two stations in Johor; Senai and Mersing. Due to the existence of some nonlinearity features in the residuals part of ARIMA modelling, two nonlinear models were introduced to capture the internal structure of the residual data. Both conventional time series models; GARCH, and machine learning model; MLP was applied to model the residuals of ARIMA model. The out-sample performance in forecasting accuracy was compared between the ARIMA-GARCH model and the ARIMA-MLP model. The findings proves that MLP model has outperformed GARCH model in capturing the dynamics in the residual data by providing the lowest error measurements. Thus, the machine learning approaches has proven its superiority against the conventional time series nonlinear model in handling the nonlinearity in the daily wind speed series for wind speed forecasting.

Enhanced Spatio-Temporal Modeling for Rainfall Forecasting: A High-Resolution Grid Analysis

Rainfall serves as a lifeline for crop cultivation in many agriculture-dependent countries including India. Being spatio-temporal data, the forecasting of rainfall becomes a more complex and tedious process. Application of conventional time series models and machine learning techniques will not be a suitable choice as they may not adequately account for the complex spatial and temporal dependencies integrated within the data. This demands some data-driven techniques that can handle the intrinsic patterns such as non-linearity, non-stationarity, and non-normality. Space–Time Autoregressive Moving Average (STARMA) models were highly known for its ability to capture both spatial and temporal dependencies, offering a comprehensive framework for analyzing complex datasets. Spatial Weight Matrix (SWM) developed by the STARMA model helps in integrating the spatial effects of the neighboring sites. The study employed a novel dataset consisting of annual rainfall measurements spanning over 50 (1970–2019) years from 119 different locations (grid of 0.25 × 0.25 degree resolution) of West Bengal, a state of India. These extensive datasets were split into testing and training groups that enable the better understanding of the rainfall patterns at a granular level. The study findings demonstrated a notable improvement in forecasting accuracy by the STARMA model that can exhibit promising implications for agricultural management and planning, particularly in regions vulnerable to climate variability.

Forecasting Stock Market Realized Volatility using Random Forest and Artificial Neural Network in South Africa

Volatility is often used as a key input into several financial models, yet there is still no consensus on the best-performing model in forecasting stock market returns volatility. Conventional time series models such as GARCH are the preferred models in the literature. However, this project aims to first adopt two novel non-linear machine learning algorithms, namely the Random Forest and Artificial Neural Network (ANN). The project then compares the performance of these two models in predicting stock market realized volatility for the JSE Basic Material Index (JBIND) and the JSE Financials Index (JFIN) over a period of five years. Based on the results of the project, the Random Forest model outperformed the ANN model for both the JFIN and JBIND index. Lastly, the COVID effect on the model’s performance was also considered and the results show that the negative impact of COVID on the model’s performance is ambiguous.

On improved fitting using a new probability distribution and artificial neural network: Application

Statistical modeling and forecasting are crucial to understanding the depth of information in data from all sources. For precision purposes, researchers are always in search of ways to improve the quality of modeling and forecasting, whatever the complexity of the situation. To this end, new (probability) distributions and suitable forecasting methods are demanded. The first part of this paper contributes to this direction. Indeed, we introduce a modified version of the flexible Weibull distribution, called the modified flexible Weibull distribution. It is constructed by mixing the flexible Weibull distribution with the exponential T-X scheme. This strategy is winning; the new distribution has a larger panel of functionalities in comparison to those of the classical Weibull distribution, among other things. To check the quality of the fitting of the modified flexible Weibull distribution, two different datasets are analyzed. After analyzing these datasets, it is observed that the modified flexible Weibull distribution has improved fitting power compared to other similar distributions. Apart from this, the conventional time series model, namely, the autoregressive integrated moving average (ARIMA) model, and the modern artificial neural network (ANN) model are considered for forecasting results. Utilizing the two datasets discussed earlier, it was discovered that the ANN model is more effective than the traditional ARIMA model.

On the update frequency of univariate forecasting models

In univariate time series forecasting, models are typically updated at every single review period. This practice, which includes specifying the optimal form of the model and estimating its parameters, theoretically allows the models to exploit new information and to respond quickly to possible structural breaks. We argue that such updates may be irrelevant in practice, also unnecessarily increasing computational cost and forecast instability. Using two large data sets of monthly and daily series as well as an indicative family of conventional time series models, we investigate several model updating scenarios, ranging from complete model form specification and parameter estimation at every review period to no updating at all. We find that intermediate updating scenarios, including the re-estimation of specific parameters but not necessarily the specification of the model form, can result in similar or even better accuracy with significantly lower computational cost. We also show that similar conclusions hold true for popular machine learning methods, as well as for setups where different approaches are utilized for training the models or accelerating their specification and estimation. We discuss the implications of our findings for manufacturers, suppliers, and retailers and propose avenues for future advances in the area of model frequency updating.

Bayesian Modeling of Labor Earnings in Construction

Labor costs constitute a significant portion of construction costs. Reliable forecasts provide insight into the movements of labor costs and are critical to the success of projects. Past studies have primarily focused on forecasting construction cost indexes or material costs. Only a few studies have concentrated on forecasting construction labor costs. This study presents a multivariate Bayesian structural time series (MBSTS) model to characterize the future values of construction labor’s average hourly earnings (AHE) using a set of candidate predictors. The methodologies commonly used by past studies do not adequately address the uncertainties associated with the modeling process. In contrast, MBSTS recognizes the uncertainty in its modeling process, which enables practitioners to quantify and account for future labor cost risks in decisions. Furthermore, the MBSTS method results in a transparent model, helping analysts investigate the rationality of the parameters. This article trains MBSTS models under four different data subset lengths (i.e., 150, 144, 138, and 132 months) to study the consistency of the explanatory variables and their corresponding coefficients. The analysis results indicate that the gross domestic product (GDP), housing starts (HS), number of building permits (BP), Construction Cost Index (CCI), Dow Jones Industrial Average (DJI), and Standard and Poor’s 500 index (SPI) are the most frequently used predictors in the regression component of the MBSTS models. The results indicate an inverse relationship between AHE from one side and HS and BP from the other. Likewise, a direct relationship exists between AHE and GDP, CCI, DJI, and SPI. The MBSTS model performed well on the validation subset in the midrange prediction intervals (i.e., 12- and 18-month periods). However, it was outperformed by conventional time series models [i.e., seasonal autoregressive integrated moving average (SARIMA)] when used for short-term forecasting. The proposed framework can be applied to facilitate monetary resource allocation in projects.

Evaluation of Weather Information for Short-Term Wind Power Forecasting with Various Types of Models

The rising share of renewable energy in the energy mix brings with it new challenges such as power curtailment and lack of reliable large-scale energy grid. The forecasting of wind power generation for provision of flexibility, defined as the ability to absorb and manage fluctuations in the demand and supply by storing energy at times of surplus and releasing it when needed, is important. In this study, short-term forecasting models of wind power generation were developed using the conventional time-series method and hybrid models using support vector regression (SVR) based on rolling origin recalibration. For the application of the methodology, the meteorological database from Korea Meteorological Administration and actual operating data of a wind power turbine (2.3 MW) from 1 January to 31 December 2015 were used. The results showed that the proposed SVR model has higher forecasting accuracy than the existing time-series methods. In addition, the conventional time-series model has high accuracy under proper curation of wind turbine operation data. Therefore, the analysis results reveal that data curation and weather information are as important as the model for wind power forecasting.

Re-visiting the COVID-19 analysis using the class of high ordered integer-valued time series models with harmonic features

The COVID-19 series is obviously one of the most volatile time series with lots of spikes and oscillations. The conventional integer-valued auto-regressive time series models (INAR) may be limited to account for such features in COVID-19 series such as severe over-dispersion, excess of zeros, periodicity, harmonic shapes and oscillations. This paper proposes alternative formulations of the classical INAR process by considering the class of high-ordered INAR models with harmonic innovation distributions. Interestingly, the paper further explores the bivariate extension of these high-ordered INARs. South Africa and Mauritius’ COVID-19 series are re-scrutinized under the optic of these new INAR processes. Some simulation experiments are also executed to validate the new models and their estimation procedures.

A sparse multivariate time series model-based fault detection method for gearboxes under variable speed condition

Gearboxes often operate under variable speed condition which makes the collected vibration signal, a widely employed type of condition monitoring data, becomes non-stationary. This paper proposes a sparse linear parameter varying vector auto-regression (LPV-VAR) model-based method for fault detection of gearboxes under variable speed condition. The proposed sparse LPV-VAR model is a multivariate time-variant time series model that can represent multichannel non-stationary baseline vibration signals from a gearbox. Fault detection is based on the residuals of the sparse LPV-VAR model. The proposed sparse LPV-VAR model inherits the strengths of the sparse time series modeling and utilization of multichannel vibration signals, where the former has shown to have higher modeling accuracy than conventional non-sparse time series models, and the latter enables the removal of the correlated random noise between channels. Both simulation and experimental studies have been conducted to validate the fault detection performance of the proposed method. Results have shown that the sparse LPV-VAR model has higher modeling accuracy than the reported sparse single-variate LPV-AR and conventional non-sparse LPV-VAR models. Subsequently, the sparse LPV-VAR model-based fault detection method achieves a higher fault detection rate than using the other two models.

An endemic–epidemic beta model for time series of infectious disease proportions

Time series of proportions of infected patients or positive specimens are frequently encountered in disease control and prevention. Since proportions are bounded and often asymmetrically distributed, conventional Gaussian time series models only apply to suitably transformed proportions. Here we borrow both from beta regression and from the well-established HHH model for infectious disease counts to propose an endemic–epidemic beta model for proportion time series. It accommodates the asymmetric shape and heteroskedasticity of proportion distributions and is consistent for complementary proportions. Coefficients can be interpreted in terms of odds ratios. A multivariate formulation with spatial power-law weights enables the joint estimation of model parameters from multiple regions. In our application to a flu activity index in the USA, we find that the endemic–epidemic beta model provides a better fit than a seasonal ARIMA model for the logit-transformed proportions. Furthermore, a multivariate approach can improve regional forecasts and reduce model complexity in comparison to univariate beta models stratified by region.

Evaluating a seasonal autoregressive moving average model with an exogenous variable for short-term timber price forecasting

In this paper roundwood prices were forecasted by means of the following time series models: (1) classical univariate autoregressive moving average (ARIMA), (2) seasonal univariate (SARIMA), and (3) seasonal bivariate with an exogenous variable (SARIMAX). The data consisted of time series of the nominal prices of pine, spruce, beech, birch, and alder roundwood and the construction confidence index (CCI) in Poland for the years 2005–2020. The results indicate that roundwood prices can be evaluated by the implemented time series models, with forecast accuracy depending on the time horizon, the model applied, and tree species. The incorporation of a separate component directly describing seasonal fluctuations into a conventional ARIMA time series model significantly improved the accuracy of roundwood price forecasts exhibiting seasonal variation. The CCI lagged by one to three quarters were found to be significantly correlated with the prices of roundwood. A SARIMAX model incorporating the CCI as a leading variable was able to account for changes in the business cycle, thus increasing the accuracy of short-term forecasts of the prices of roundwood used in the construction industry.

Impact Study of Temperature on the Time Series Electricity Demand of Urban Nepal for Short-Term Load Forecasting

Short-term electricity demand forecasting is one of the best ways to understand the changing characteristics of demand that helps to make important decisions regarding load flow analysis, preventing imbalance in generation planning, demand management, and load scheduling, all of which are actions for the reliability and quality of that power system. The variation in electricity demand depends upon various parameters, such as the effect of the temperature, social activities, holidays, the working environment, and so on. The selection of improper forecasting methods and data can lead to huge variations and mislead the power system operators. This paper presents a study of electricity demand and its relation to the previous day’s lags and temperature by examining the case of a consumer distribution center in urban Nepal. The effect of the temperature on load, load variation on weekends and weekdays, and the effect of load lags on the load demand are thoroughly discussed. Based on the analysis conducted on the data, short-term load forecasting is conducted for weekdays and weekends by using the previous day’s demand and temperature data for the whole year. Using the conventional time series model as a benchmark, an ANN model is developed to track the effect of the temperature and similar day patterns. The results show that the time series models with feedforward neural networks (FF-ANNs), in terms of the mean absolute percentage error (MAPE), performed better by 0.34% on a weekday and by 8.04% on a weekend.

Using spatio‐temporal deep learning for forecasting demand and supply‐demand gap in ride‐hailing system with anonymised spatial adjacency information

To reduce passenger waiting time and driver search friction, ride-hailing companies need to accurately forecast spatio-temporal demand and supply-demand gap. However, due to spatio-temporal dependencies pertaining to demand and supply-demand gap in a ride-hailing system, making accurate forecasts for both demand and supply-demand gap is a difficult task. Furthermore, due to confidentiality and privacy issues, ride-hailing data are sometimes released to the researchers by removing spatial adjacency information of the zones, which hinders the detection of spatio-temporal dependencies. To that end, a novel spatio-temporal deep learning architecture is proposed in this paper for forecasting demand and supply-demand gap in a ride-hailing system with anonymized spatial adjacency information, which integrates feature importance layer with a spatio-temporal deep learning architecture containing one-dimensional convolutional neural network (CNN) and zone-distributed independently recurrent neural network (IndRNN). The developed architecture is tested with real-world datasets of Didi Chuxing, which shows that our models based on the proposed architecture can outperform conventional time-series models (e.g., ARIMA) and machine learning models (e.g., gradient boosting machine, distributed random forest, generalized linear model, artificial neural network). Additionally, the feature importance layer provides an interpretation of the model by revealing the contribution of the input features utilized in prediction.

Deep Sequence Modeling: Development and Applications in Asset Pricing

We predict asset returns and measure risk premia using a prominent technique from artificial intelligence -- deep sequence modeling. Because asset returns often exhibit sequential dependence that may not be effectively captured by conventional time series models, sequence modeling offers a promising path with its data-driven approach and superior performance. In this paper, we first overview the development of deep sequence models, introduce their applications in asset pricing, and discuss their advantages and limitations. We then perform a comparative analysis of these methods using data on U.S. equities. We demonstrate how sequence modeling benefits investors in general through incorporating complex historical path dependence, and that Long- and Short-term Memory (LSTM) based models tend to have the best out-of-sample performance.

Annual Precipitation in Southern of Madagascar: Modeling using High Order Fuzzy Time Series

The objective of this research is to find the best conventional high order fuzzy time series model for annual precipitation series in southern Madagascar. This work consists on finding the hyper parameters (number of partition of the universe of discourse and model order) to obtain the best conventional high order fuzzy time series model for our experimental data. In previous works, entitled spatial and temporal variability of precipitation in southern Madagascar, we subdivided the study area between 22 ° S to 30 ° S latitude and 43 ° Eto 48 ° E longitude into four zones of homogeneous precipitation. In this article, we seek to model annual precipitation data representative of one of these four areas. These data were taken between 1979 and 2017. Our approach consists on subdividing the data: data obtained from 1979 to 2001 (60%) for the training and data from 2002 to 2017 (40%) to test the model. To determine the number of partitions and model order, we fix first the number of partitions to 10 and then to 15, 20, 25,30, 35, 40, 45 and 50.For each of these values, we vary the model order from 1 to 10.Thenwe locate the model order which corresponds to the minimum of the average curve between the Mean Absolute Errors (MAE) between the training data and the test data. Thus, the orders of the candidate model are 2, 3, 5, and 6.The next step is to fix the model order with the previous values and vary the number of partitions from 3 to 50.For each couple of hyper parameter of the model (number of partitions, model order), we locate the value of number of partitions corresponding to the minimum of the average curve between the absolute mean of the errors or MAE (Mean Absolute Error) between the train and test data. We obtain the hyper-parameter pairs (37, 2), (20, 3), (35, 5) and (35, 6).The first pair gives the lowest Mean Absolute Error. As a final result, we obtain the best high order fuzzy time series model with hyperparameters number of partition equals thirty seven and of order equals two for annual precipitation in Southern of Madagascar

Neural Network Models for Empirical Finance

This paper presents an overview of the procedures that are involved in prediction with machine learning models with special emphasis on deep learning. We study suitable objective functions for prediction in high-dimensional settings and discuss the role of regularization methods in order to alleviate the problem of overfitting. We also review other features of machine learning methods, such as the selection of hyperparameters, the role of the architecture of a deep neural network for model prediction, or the importance of using different optimization routines for model selection. The review also considers the issue of model uncertainty and presents state-of-the-art methods for constructing prediction intervals using ensemble methods, such as bootstrap and Monte Carlo dropout. These methods are illustrated in an out-of-sample empirical forecasting exercise that compares the performance of machine learning methods against conventional time series models for different financial indices. These results are confirmed in an asset allocation context.

Enhanced fuzzy time series forecasting model based on hesitant differential fuzzy sets and error learning

Most of existing fuzzy time series forecasting models lead to unsatisfactory forecasting performance due to deficiencies in constructing fuzzy intervals. This paper adopts the Fuzzy Silhouette criterion to determine the optimal number and length of fuzzy intervals effectively and objectively. The universe of discourse with equal and unequal intervals are merged by aggregating hesitant information. Second, multiple relevant factors are considered to reveal the complex attributes of the real-world data, and differential fuzzy sets are included to reconnoiter deep trends. Finally, an optimal error learning mechanism is applied to enhance the forecasting performance. Key strengths of the developed model are verifying the possibility of increasing the performance of conventional fuzzy time series models by integrating them with new components and processes. Three typical stock index datasets are selected to evaluate the performance of the proposed model and experimental results prove that it outperforms compared models in forecasting accuracy and profitable ability.

Breakpoint detection in non-stationary runoff time series under uncertainty

Most conventional hydrological time series models, used for forecasting or synthetic data generation, are based on the stationary hypothesis. Hydrologic variables such as surface runoff discharge exhibit non-stationary behavior due to the effects of climate change or human activities. In this paper, a new methodology is proposed to analyze the uncertainty of the number and location of breakpoints in runoff time series. The non-stationary runoff time series is simulated by segmenting the series into stationary pieces, which follow an Auto-Regressive Moving Average (ARMA) process, based on the minimum description length (MDL) principle. The length of each piece and the order of the piecewise ARMA model are optimized using the Genetic Algorithm (GA). Several runoff time series with variable lengths are generated based on the observed data and the probability distribution function(s) (PDFs) of breakpoint location(s) are constructed. An optimal set of breakpoint locations are found for each generated runoff time series by minimizing the objective function based on the MDL. Finally, the PDF(s) of breakpoint location(s) is (are) constructed using the results of the mentioned optimization model. To evaluate the proposed methodology, it is applied to the case study of Zayandehrud River in Iran. According to the obtained breakpoint PDF for the runoff time series, three breakpoints around years of 1987, 1996 and 2006 are identified. The results are analyzed by investigating the time series of agricultural lands derived using remote sensing data and studying the impacts of interbasin water transfer projects implemented in the study area. Overall, investigating human-induced changes in the study area confirms the obtained shape of the PDF of runoff breakpoint and shows the good performance of the developed method for finding the number and location of breakpoints in the runoff time series.

Forecasting the patterns of COVID-19 and causal impacts of lockdown in top five affected countries using Bayesian Structural Time Series Models.

There are numerous studies dealing with analysis for the future patterns of COVID-19 in different countries using conventional time series models. This study aims to provide more flexible analytical framework that decomposes the important components of the time series, incorporates the prior information, and captures the evolving nature of model parameters. We have employed the Bayesian structural time series (BSTS) models to investigate the temporal dynamics of COVID-19 in top five affected countries around the world in the time window March 1, 2020 to June 29, 2020. In addition, we have analyzed the casual impact of lockdown in these countries using intervention analysis under BSTS models. We achieved better levels of accuracy as compared to ARIMA models. The forecasts for the next 30 days suggest that India, Brazil, USA, Russia and UK are expected to have 101.42%, 85.85%, 46.73%, 32.50% and 15.17% increase in number of confirmed cases, respectively. On the other hand, there is a chance of 70.32%, 52.54%, 45.65%, 19.29% and 18.23% growth in the death figures for India, Brazil, Russia, USA and UK, respectively. In addition, USA and UK have made quite sagacious choices for lifting/relaxing the lockdowns. However, the pace of outbreak has significantly increased in Brazil, India and Russia after easing the lockdowns. On the whole, the Indian and Brazilian healthcare system is likely to be seriously overburdened in the next month. Though USA and Russia have managed to cut down the rates of positive cases, but serious efforts will be required to keep these momentums on. On the other hand, UK has been successful in flattening their outbreak trajectories.