Time Series Problems Research Articles

SVM kernel based on particle swarm optimized vector and Bayesian optimized SVM in atmospheric particulate matter forecasting

The application of Artificial Intelligence (AI) has been upgraded in many scientific fields the last years, with the development of new artificial intelligence-based technologies and techniques. Considering that in the literature there is a very limited number of studies proposing and testing new SVM kernels in regression problems, this research introduces a novel SVM Kernel by incorporating a transformed particle swarm optimized ANN weight vector in a Bayesian optimized SVM kernel in a time series problem for predicting the atmospheric pollutant factor Particulate Matter 10 (PM10). The proposed model introduces a new SVM kernel that illustrates an increased forecasting accuracy compared to the conventional optimized ANN and SVM models according to the experimental results. The findings of the proposed methodology illustrate that the new proposed SVM Kernel can be utilized as an improved forecasting technique.

Towards activation function search for long short-term model network: A differential evolution based approach

In Deep Neural Networks (DNNs), several architectures had been proposed for the various complex tasks such as Machine Translation, Natural Language processing and time series forecasting. Long-Short Term Model (LSTM), a deep neural network became the popular architecture for solving sequential and time series problems and achieved markable results. On building the LSTM model, many hyper-parameters like activation function, loss function, and optimizer need to be set in advance. These hyper-parameters play a significant role in the performance of the DNNs. This work concentrates on finding a novel activation function that can replace the existing activation function such as sigmoid and tanh in the LSTM. The Differential Evolution Algorithm (DEA) based search methodology is proposed in our work to discover the novel activation function for the LSTM network. Our proposed methodology finds an optimal activation function that outperforms than the traditional activation functions like sigmoid (σ), hyperbolic tangent (tanh) and Rectified Linear Unit (ReLU). In this work, the newly explored activation function based on DEA methodology is sinh(x)+sinh-1(x) named as Combined Hyperbolic Sine (comb-H-sine) function. The proposed comb-H-sine activation function outperforms the traditional functions in LSTM with accuracy of 98.83%,93.49% and 78.38% with MNIST, IMDB and UCI HAR datasets respectively.

Time series forecasting by global optimization method

The actual problem of time series forecasting associated with the choice of the type of forecasting functions is considered. The aim of the work is to develop a new forecasting method, in which, along with the construction of the predictive function, the values of the predicted characteristics are directly optimized at the entire depth of the forecast. The developed method belongs to the class of probabilistic statistical forecast methods and partially to the class of feedback methods. Typical representatives of the methods of these classes are correlation and regression analysis, factor and variance analysis, statistical modeling. In the new method the predictive function is constructed in the basis of trigonometric polynomials, which leads to multi extremality of the forecasting problem. In this regard, to solve the problem of forecasting, the method of global optimization, hereinafter called the criterion shifts method, is used. The developed forecasting method allows solving a wide range of practical forecasting problems. Since the method of global solution search is used, it guarantees the exact finding of the predicted characteristics obtained as a result of their optimization by the criterion of regularity. The results obtained in this article showed high accuracy.

Neural network approach to forecasting the cost of ferroalloy products

The article notes the increasing role of ferroalloy sub-sector in the qualitative development of metallurgy. Progress predicts of modern metallurgy are difficult in the context of increasing risks of global economic development. The high volatility of domestic producers’ prices for the main ferroalloys also has a negative impact. It is necessary to develop methodological tools for forecasting changes in market prices for metallurgical products with a high degree of accuracy. One of the important areas of application in metallurgy forecasting tools is construction of a model for forecasting the cost of ferroalloy products. It is the main purpose of the study. On the example of constructing a forecast model for changing the price of ferrosilicon, relevance of the neural network approach to forecasting the cost of ferroalloy products was substantiated. As part of the tasks of industry development, the capabilities of neural networks have been poorly studied to date. Formal description of the time series forecasting model based on neural networks is given. When constructing neural networks, any time series problem is represented as a multidimensional regression problem. The main parameters of predictive networks training are highlighted. The average price of ferrosilicon on the Russian market and the prices in the Russian regions were used as input variables. The networks that meet the qualitative criteria of forecasting models were trained. Selection of the networks was carried out taking into account the results of graphical analysis and cross-checking. A neural network model was constructed to predict the change in ferrosilicon price in the short term with high accuracy. This model can be useful in strategic decisions justifying in the activities of industry research institutes and metallurgical enterprises.

Effective long short-term memory with fruit fly optimization algorithm for time series forecasting

A number of recent studies have adopted long short-term memory (LSTM) in extensive applications, such as handwriting recognition and time series prediction, with considerable success. However, the parameters of LSTM have greatly influenced its accuracy and performance. In this study, LSTM with fruit fly optimization algorithm (FOA), called FOA-LSTM, is designed to solve time series problems. As a novel intelligent algorithm, FOA is applied to decide on the optimal hyper-parameter of LSTM. Experiments under the NN3 time series, three comparative experiments and the monthly energy consumption of the USA are conducted to verify the effectiveness of the FOA-LSTM model. The results indicate that the symmetric mean absolute percentage error (SMAPE) is reduced by up to 11.44% in the last 11 monthly series in the NN3 dataset. Four comparative experiments and the real-life series verify further that the FOA-LSTM model obtains a better result compared with other forecasting models.

Study of LSTM Air Quality Index Prediction Based on Forecasting Timeliness

The change of urban air quality is affected by pollutant emission, meteorological conditions and other factors, so air quality prediction is a multi-variable, nonlinear and time-series problem, which is difficult to be predicted by traditional methods. To solve this problem, a circular neural network based on Long Short Time Memory (LSTM) is proposed to predict Air Quality Index (AQI) by considering the pollution sources, meteorological conditions and time series. The transfer entropy is used to select the meteorological factors that affect the strong change of AQI. Combined with the prediction time, the prediction accuracy of this algorithm in the future 0∼48 hours within different forecast time is studied and compared with the traditional BP neural network and the Gated Recurrent Unit (GRU), and then the Root Mean Square Error (RMSE) was used for evaluation. Taking the measured data of hourly air quality index of Chengdu from January 1, 2018 to September 15, 2019 and the measured data of meteorological factors in the same period as experimental examples, the experimental results show that LSTM has better prediction accuracy and robustness than traditional neural networks in the aging of 0∼48h forecasting, and has the advantages of temporary forecasting and short-term forecasting. At the same time, it is verified that GRU has no obvious advantage in air quality index prediction application compared with LSTM.

Interval Type-2 Intuitionistic Fuzzy Logic System for Time Series and Identification Problems - A Comparative Study

This paper proposes a sliding mode control-based learning of interval type-2 intuitionistic fuzzy logic system for time series and identification problems. Until now, derivative-based algorithms such as gradient descent back propagation, extended Kalman filter, decoupled extended Kalman filter and hybrid method of decoupled extended Kalman filter and gradient descent methods have been utilized for the optimization of the parameters of interval type-2 intuitionistic fuzzy logic systems. The proposed model is based on a Takagi-Sugeno-Kang inference system. The evaluations of the model are conducted using both real world and artificially generated datasets. Analysis of results reveals that the proposed interval type-2 intuitionistic fuzzy logic system trained with sliding mode control learning algorithm (derivative-free) do outperforms some existing models in terms of the test root mean squared error while competing favourable with other models in the literature. Moreover, the proposed model may stand as a good choice for real time applications where running time is paramount compared to the derivative-based models.

An iterative end point fitting based trend segmentation representation of time series and its distance measure

Symbolic approximation representation is a key problem in time series which can significantly affect the accuracy and efficiency of data mining. However, since currently used methods divide the original sequence into segments with equal size, they ignore one of the most important features of time series: the trend. To overcome the defect of equal-sized segmenting, we present a trend segmentation representation based on Iterative End Point Fitting algorithm (IEPF-TSR). Particularly, we use iterative end point fitting (IEPF) algorithm to search the break point of each segment and get the trend segmentation. Then a triplet based symbolic representation is proposed for each segment which includes the start point, mean and trend. Moreover, we define a new distance measure method based on trend segmentation representation (TSR-DIST) which can suit for two representations with different lengths, and prove it to be the lower bound of Euclidean distance. The experimental results on UCR datasets show that the proposed representation and distance measure achieve better performance than the state-of-the-art methods in the classification accuracy and the dimensionality reduction ratio.

Temporal rainfall disaggregation using a micro-canonical cascade model: possibilities to improve the autocorrelation

Abstract. In urban hydrology rainfall time series of high resolution in time are crucial. Such time series with sufficient length can be generated through the disaggregation of daily data with a micro-canonical cascade model. A well-known problem of time series generated in this way is the inadequate representation of the autocorrelation. In this paper two cascade model modifications are analysed regarding their ability to improve the autocorrelation in disaggregated time series with 5 min resolution. Both modifications are based on a state-of-the-art reference cascade model (method A). In the first modification, a position dependency is introduced in the first disaggregation step (method B). In the second modification the position of a wet time step is redefined in addition by taking into account the disaggregated finer time steps of the previous time step instead of the previous time step itself (method C). Both modifications led to an improvement of the autocorrelation, especially the position redefinition (e.g. for lag-1 autocorrelation, relative errors of −3 % (method B) and 1 % (method C) instead of −4 % for method A). To ensure the conservation of a minimum rainfall amount in the wet time steps, the mimicry of a measurement device is simulated after the disaggregation process. Simulated annealing as a post-processing strategy was tested as an alternative as well as an addition to the modifications in methods B and C. For the resampling, a special focus was given to the conservation of the extreme rainfall values. Therefore, a universal extreme event definition was introduced to define extreme events a priori without knowing their occurrence in time or magnitude. The resampling algorithm is capable of improving the autocorrelation, independent of the previously applied cascade model variant (e.g. for lag-1 autocorrelation the relative error of −4 % for method A is reduced to 0.9 %). Also, the improvement of the autocorrelation by the resampling was higher than by the choice of the cascade model modification. The best overall representation of the autocorrelation was achieved by method C in combination with the resampling algorithm. The study was carried out for 24 rain gauges in Lower Saxony, Germany.

Sufficient conditions for the existence of a sample mean of time series under dynamic time warping

Time series averaging is an important subroutine for several time series data mining tasks. The most successful approaches formulate the problem of time series averaging as an optimization problem based on the dynamic time warping (DTW) distance. The existence of an optimal solution, called sample mean, is an open problem for more than four decades. Its existence is a necessary prerequisite to formulate exact algorithms, to derive complexity results, and to study statistical consistency. In this article, we propose sufficient conditions for the existence of a sample mean. A key result for deriving the proposed sufficient conditions is the Reduction Theorem that provides an upper bound for the minimum length of a sample mean.

A Rainfall‐Runoff Model With LSTM‐Based Sequence‐to‐Sequence Learning

AbstractRainfall‐runoff modeling is a complex nonlinear time series problem. While there is still room for improvement, researchers have been developing physical and machine learning models for decades to predict runoff using rainfall data sets. With the advancement of computational hardware resources and algorithms, deep learning methods such as the long short‐term memory (LSTM) model and sequence‐to‐sequence (seq2seq) modeling have shown a good deal of promise in dealing with time series problems by considering long‐term dependencies and multiple outputs. This study presents an application of a prediction model based on LSTM and the seq2seq structure to estimate hourly rainfall‐runoff. Focusing on two Midwestern watersheds, namely, Clear Creek and Upper Wapsipinicon River in Iowa, these models were used to predict hourly runoff for a 24‐hr period using rainfall observation, rainfall forecast, runoff observation, and empirical monthly evapotranspiration data from all stations in these two watersheds. The models were evaluated using the Nash‐Sutcliffe efficiency coefficient, the correlation coefficient, statistical bias, and the normalized root‐mean‐square error. The results show that the LSTM‐seq2seq model outperforms linear regression, Lasso regression, Ridge regression, support vector regression, Gaussian processes regression, and LSTM in all stations from these two watersheds. The LSTM‐seq2seq model shows sufficient predictive power and could be used to improve forecast accuracy in short‐term flood forecast applications. In addition, the seq2seq method was demonstrated to be an effective method for time series predictions in hydrology.

Long Short-Term Memory Networks for Traffic Flow Forecasting: Exploring Input Variables, Time Frames and Multi-Step Approaches

Traffic flow forecasting is an acknowledged time series problem whose solutions have been essentially grounded on statistical-based models. Recent times came, however, with promising results regarding the use of Recurrent Neural Networks (RNNs), such as Long Short-Term Memory networks (LSTMs), to accurately address time series problems. Literature is, however, evasive in regard to several aspects of the conceived models and often exhibits misconceptions that may lead to important pitfalls. This study aims to conceive and find the best possible LSTM model for traffic flow forecasting while addressing several important aspects of such models such as the multitude of input features, the time frames used by the model and the employed approach for multi-step forecasting. To overcome the spatial problem of open source datasets, this study presents and describes a new dataset collected by the authors of this work. After several weeks of model fitting, Recursive Multi-Step Multi-Variate models were the ones showing better performance, strengthening the perception that LSTMs can be used to accurately forecast the traffic flow for several future timesteps.

Gaussian Process Regression Ensemble Model for Network Traffic Prediction

Network traffic prediction is substantial for network optimization and resource management. However, designing an efficient predictive model considering different traffic characteristics, including periodicity, nonlinearity, and nonstationarity, is challenging. Recently, ensemble learning is attracting much attention from researchers in the machine learning community. Although ensemble learning has proven exceptional performance in modelling the intricate problems, it may not be able to handle varying patterns and chaotic behaviour, which are typical properties of traffic data (and many other time-series problems). For this reason, ensemble methods show a limited prediction accuracy in network traffic modelling. We address this issue by proposing an ensemble of learners for time-series prediction that considers the accuracy of individual learners as well as diversity among their outcomes. Each learner contributes to the optimization process by finding the optimal accuracy-diversity balance in a segment of feature space. This divide-and-conquer approach avoids complicated objective functions with many local optimums while fitting the ensemble model on large datasets. Experimental results on the real traffic traces show our proposed method outperforms other state-of-the-art predictors with 12% on average in prediction accuracy for different datasets.

An Attention-Based User Preference Matching Network for Recommender System

Click-through rate prediction (CTR) is an essential task in recommender system. The existing methods of CTR prediction are generally divided into two classes. The first class is focused on modeling feature interactions, the second class is focused on solving time-series problems. However, the existing models of the second class are not able to handle time-series problems with user feedback information, so we propose PMN to solve this kind of problem. To be able to take full advantage of historical user behavior along with the user feedback, PMN uses the attention mechanism to get the user historical behavior representation and the user preference representation from the original input. Specially, user preference representation is derived from the user feedback information and it explicitly shows the user's attitude towards the candidate, which greatly improve the model performance. Finally, we introduced user preference baselines to solve the problem of inconsistent scoring standards for different users. In this paper, we focus on the CTR prediction modeling in the scenario of video recommendation in Video On Demand (VOD) service. Experimental results on multiple data sets have shown that our PMN model is effective.

Neuroevolution Strategy for Time Series Prediction

Optimization is a concept, a process, and a method that all people use on a daily basis to solve their problems. The source of many optimization methods for many scientists has been the nature itself and the mechanisms that exist in it. Neural networks, inspired by the neurons of the human brain, have gained a great deal of recognition in recent years and provide solutions to everyday problems. Evolutionary algorithms are known for their efficiency and speed, in problems where the optimal solution is found in a huge number of possible solutions and they are also known for their simplicity, because their implementation does not require the use of complex mathematics. The combination of these two techniques is called neuroevolution. The purpose of the research is to combine and improve existing neuroevolution architectures, to solve time series problems. In this research, we propose a new improved strategy for such a system. As well as comparing the performance of our system with an already existing system, competing with it on five different datasets. Based on the final results and a combination of statistical results, we conclude that our system manages to perform much better than the existing system in all five datasets.

Time series classification based on arima and adaboost

In this paper, a novel time series classification approach, which using auto regressive integrated moving average model (ARIMA) features and Adaptive Boosting (AdaBoost) classifications. ARIMA is particularly suitable for distinguishing time series signal and Adaboost is suitable for features classification. The simulation results have shown that the algorithm is feasible. And this method is more accurate than many existing method in multiple time series problems.

Context-Aware Traffic Prediction Framework Based on Series Decomposition

Forecasting traffic flow is a typical time series problem, which has attracted increasing attention due to the urgent need in intelligent transportation systems. Although numerous time series forecasting methods have been investigated in past decades, from statistics based models to deep neural networks models, the main disadvantages of aforementioned work could be summarized as follows: 1) incapable to handle the complexity and uncertainty of series; 2) incapable to consider external features such as spatial information and importance of points during the learning process; 3) unstable performance on forecasting task given various data patterns. In this study, a novel strategy was proposed to extract context-awareness information and then integrated with Temporal Convolution Network(TCN) model, namely Context-Aware Temporal Convolution Network(CATCN), which utilized local sub-segments to portrait the potential patterns of a series based on series decomposition. The experiments were conducted using three sets of field-captured traffic datasets. The results were presented and compared to state-of-the-art methodologies. The results showed that the performance of proposed method is significantly improved, especially, on the auto-correlation series corpora.

Hetero-Dimensional Multitask Neuroevolution for Chaotic Time Series Prediction

Chaotic time series prediction has important research and application value, and neural network-based prediction methods have problems such as low accuracy and difficulty in determining the number of nodes in the hidden layer. In recent years, as a representative of evolutionary multitask optimization algorithms(EMTO), multifactorial evolutionary algorithms (MFEA) have achieved excellent results in Multitask Optimization (MTO) problems, but it does not work well in solving hetero-dimensional multitask problem. We analyzed this phenomenon and proposed the concept of harmful transfer. Then to solve these problems, in this paper, we propose the hetero-dimensional multitask neuroevolution algorithms (HD-MFEA neuroevolution) for chaotic time series prediction. First, we propose hetero-dimensional assortative mating algorithm and self-adaption elite replacement algorithm, so that the hetero-dimensional multitask evolutionary algorithm (HD-MFEA) can overcome the problems caused by harmful transfer on hetero-dimensional multitask problem. Second, in order to adapt to the hierarchical characteristics of neural networks, we propose the neural level segmental coding strategy. Third, we consider the optimization of neural networks with different hidden layer nodes as hetero-dimensional multitask problem and use the HD-MFEA neuroevolution to solve this problem. We verify the effectiveness of the HD-MFEA on six hetero-dimensional multitask benchmark problems. The Mackey-Glass, Lorenz and Sunspot time series are used to demonstrate the performance of the HD-MFEA neuroevolution. Experimental results have showed that the HD-MFEA neuroevolution performs well in predicting the chaotic time series problems.

Rainfall Monitoring Based on Machine Learning by Earth-Space Link in the Ku Band

Recently, the oblique earth-space links (OELs) between satellite and earth station have been used for rainfall monitoring as a supplement to existing observation methods. Most recent studies achieved the rainfall measurement by OELs based on the empirical method such as power-law (PL) model. In practice, two crucial issues need to be addressed: 1) identification of rain and no-rain periods; and 2) determination of attenuation baseline. To solve these problems, this article adopts several machine learning algorithms based on the analysis of earth-space link signal characteristics. For the first issue, we choose the support vector machine as a classifier and the adaptive synthetic sampling algorithm is deployed to eliminate the effects caused by the data imbalance. For the second issue, the long short-term neural network is selected for the determination of attenuation baseline since it has a good ability to solve time-series problem. In terms of the rainfall inversion, we establish a new model by combining the back-propagation (BP) network and genetic algorithm (GA). The PL model is also used as a comparison. To validate the proposed method, we set up an earth-space link that receives the signal from AsiaSat5 in 12.32 GHz. The results demonstrate that the two issues are successfully addressed and the inversion of precipitation is also carried out. Compared to disdrometer, the correlation and mean absolute error of GA-BP model are 0.83 and 1.30 mm/h, respectively, indicating a great potential to use densely OELs for global precipitation monitoring.

Cross-temporal aggregation: Improving the forecast accuracy of hierarchical electricity consumption

Achieving high accuracy in energy consumption forecasting is critical for improving energy management and planning. However, this requires the selection of appropriate forecasting models, able to capture the individual characteristics of the series to be predicted, which is a task that involves a lot of uncertainty. When hierarchies of load from different sources are considered together, the uncertainty and complexity increase further. For example, when forecasting both at system and region level, not only the model selection problem is expanded to multiple time series, but we also require aggregation consistency of the forecasts across levels. Although hierarchical forecasting, such as the bottom-up, the top-down, and the optimal reconciliation methods, can address the aggregation consistency concerns, it does not resolve the model selection uncertainty. To address this issue, we rely on Multiple Temporal Aggregation (MTA), which has been shown to mitigate the model selection problem for low-frequency time series. We propose a modification of the Multiple Aggregation Prediction Algorithm, a special implementation of MTA, for high-frequency time series to better handle the undesirable effect of seasonality shrinkage that MTA implies and combine it with conventional cross-sectional hierarchical forecasting. The impact of incorporating temporal aggregation in hierarchical forecasting is empirically assessed using a real data set from five bank branches. We show that the proposed MTA approach, combined with the optimal reconciliation method, demonstrates superior accuracy, aggregation consistency, and reliable automatic forecasting.