Time Series Prediction Problems Research Articles

A Modified Adaptive Neuro-Fuzzy Inference System Using Multi-Verse Optimizer Algorithm for Oil Consumption Forecasting

Oil is the primary source of energy, therefore, oil consumption forecasting is essential for the necessary economic and social plans. This paper presents an alternative time series prediction method for oil consumption based on a modified Adaptive Neuro-Fuzzy Inference System (ANFIS) model using the Multi-verse Optimizer algorithm (MVO). MVO is applied to find the optimal parameters of the ANFIS. Then, the hybrid method, namely MVO-ANFIS, is employed to forecast oil consumption. To evaluate the performance of the MVO-ANFIS model, a dataset of two different countries was used and compared with several forecasting models. The evaluation results show the superiority of the MVO-ANFIS model over other models. Moreover, the proposed method constitutes an accurate tool that effectively improved the solution of time series prediction problems.

Generic Tracking and Probabilistic Prediction Framework and Its Application in Autonomous Driving

Accurately tracking and predicting behaviors of surrounding objects are key prerequisites for intelligent systems such as autonomous vehicles to achieve safe and high-quality decision making and motion planning. However, there still remain challenges for multi-target tracking due to object number fluctuation and occlusion. To overcome these challenges, we propose a constrained mixture sequential Monte Carlo (CMSMC) method in which a mixture representation is incorporated in the estimated posterior distribution to maintain multi-modality. Multiple targets can be tracked simultaneously within a unified framework without explicit data association between observations and tracking targets. The framework can incorporate an arbitrary prediction model as the implicit proposal distribution of the CMSMC method. An example in this paper is a learning-based model for hierarchical time-series prediction, which consists of a behavior recognition module and a state evolution module. Both modules in the proposed model are generic and flexible so as to be applied to a class of time-series prediction problems where behaviors can be separated into different levels. Finally, the proposed framework is applied to a numerical case study as well as a task of on-road vehicle tracking, behavior recognition, and prediction in highway scenarios. Instead of only focusing on forecasting trajectory of a single entity, we jointly predict continuous motions for interactive entities simultaneously. The proposed approaches are evaluated from multiple aspects, which demonstrate great potential for intelligent vehicular systems and traffic surveillance systems.

Neural Network Techniques for Time Series Prediction: A Review

It is important to predict a time series because many problems that are related to prediction such as health prediction problem, climate change prediction problem and weather prediction problem include a time component. To solve the time series prediction problem various techniques have been developed over many years to enhance the accuracy of forecasting. This paper presents a review of the prediction of physical time series applications using the neural network models. Neural Networks (NN) have appeared as an effective tool for forecasting of time series. Moreover, to resolve the problems related to time series data, there is a need of network with single layer trainable weights that is Higher Order Neural Network (HONN) which can perform nonlinearity mapping of input-output. So, the developers are focusing on HONN that has been recently considered to develop the input representation spaces broadly. The HONN model has the ability of functional mapping which determined through some time series problems and it shows the more benefits as compared to conventional Artificial Neural Networks (ANN). The goal of this research is to present the reader awareness about HONN for physical time series prediction, to highlight some benefits and challenges using HONN.

Meta-Descent for Online, Continual Prediction

This paper investigates different vector step-size adaptation approaches for non-stationary online, continual prediction problems. Vanilla stochastic gradient descent can be considerably improved by scaling the update with a vector of appropriately chosen step-sizes. Many methods, including AdaGrad, RMSProp, and AMSGrad, keep statistics about the learning process to approximate a second order update—a vector approximation of the inverse Hessian. Another family of approaches use meta-gradient descent to adapt the stepsize parameters to minimize prediction error. These metadescent strategies are promising for non-stationary problems, but have not been as extensively explored as quasi-second order methods. We first derive a general, incremental metadescent algorithm, called AdaGain, designed to be applicable to a much broader range of algorithms, including those with semi-gradient updates or even those with accelerations, such as RMSProp. We provide an empirical comparison of methods from both families. We conclude that methods from both families can perform well, but in non-stationary prediction problems the meta-descent methods exhibit advantages. Our method is particularly robust across several prediction problems, and is competitive with the state-of-the-art method on a large-scale, time-series prediction problem on real data from a mobile robot.

AT-LSTM: An Attention-based LSTM Model for Financial Time Series Prediction

This paper proposes an attention-based LSTM (AT-LSTM) model for financial time series prediction. We divide the prediction process into two stages. For the first stage, we apply an attention model to assign different weights to the input features of the financial time series at each time step. In the second stage, the attention feature is utilized to effectively select the relevant feature sequences as input to the LSTM neural network for the prediction in the next time frame. Our proposed framework not only solves the long-term dependence problem of time series prediction effectively, but also improves the interpretability of the time series prediction methods based on the neural network. In the end of this paper, we conducted experiments on financial time series prediction task with three real-world data sets. The experimental results show that our framework for time series pre-diction is state-of-the-art against the baselines.

Hierarchical fuzzy design by a multi-objective evolutionary hybrid approach

This paper presents a new tree hierarchical representation of type-2 fuzzy systems. The proposed system is called the type-2 hierarchical flexible beta fuzzy system (T2HFBFS) and is trained based on two-phase optimization mechanism. The first optimization step is a multi-objective structural learning phase. This phase is based on the multi-objective extended immune programming algorithm and aims to obtain an improved T2HFBFS structure with good interpretability-accuracy trade-off. The second optimization step is a parameter tuning phase. Using a hybrid evolutionary algorithm, this phase allows the adjustment of antecedent and consequent membership function parameters of the obtained T2HFBFS. By interleaving the two learning steps, an optimal and accurate hierarchical type-2 fuzzy system is derived with the least number of possible rules. The performance of the system is evaluated by conducting case studies for time series prediction problems and high-dimensional classification problems. Results prove that the T2HFBFS could attain superior performance than other existing approaches in terms of achieving high accuracy with a significant rule reduction.

An improved Elman neural network with piecewise weighted gradient for time series prediction

Time series prediction is an important tool for system analysis. Traditional forecasting methods usually achieve prediction by establishing static model or only mining the information from the sequence itself, but without considering the dynamics of the system or its triggering factors. Elman neural network (ENN) is a dynamical model that can remember historical states. In order to better establish the real multivariate time series model and improve the single-step prediction accuracy, this study proposes an improved regularized ENN method based on piecewise time weighted gradient (PWRENN). Each time the parameters of PWRENN model are updated, the weighted gradient is calculated by weighing the current gradient and the previous gradients according to a monotonically increasing temporal function. By considering both the current gradient and previous gradients, PWRENN is able to simulate the time-lag effect in the time series prediction problem. Moreover, the system environment may change with time, and the historical data of the same slot at a different time should have different effects on the predicted data. This work uses a piecewise time function with different parameters at different time periods to more accurately model the real system. In PWRENN, L2 regularization is adopted to solve the overfitting problem and enhance the generalization performance. Furthermore, the effectiveness of the proposed method is verified in an artificial Mackey-Glass time series prediction and three landslide displacement predictions.

Simplifying Long Short-Term Memory for Fast Training and Time Series Prediction

Long short Term Memory(LSTM) has been widely used in sequencial problems. However, for the time series prediction problems, its complex structure limits its running speed and performance. In order to solve this problem, this paper simplified the standard LSTM model by reducing the number of gates and the parameters involved in gates computation. Experiments on univariate data set and multivariate data set show that the proposed simplified model not only has better accuracy, but also has higher running speed.

SVRGSA: a hybrid learning based model for short‐term traffic flow forecasting

Accurate and timely short-term traffic flow forecasting is a critical component for intelligent transportation systems. However, it is quite challenging to develop an efficient and robust forecasting model due to complex non-linear data pattern of traffic flow. Support vector regression (SVR) has been widely employed in non-linear regression and time series prediction problems. However, the lack of knowledge of the choice of hyper-parameters in the SVR model leads to poor forecasting accuracy. In this study, the authors propose a hybrid traffic flow forecasting model combining gravitational search algorithm (GSA) and the SVR model. The GSA is employed to search optimal SVR parameters. Extensive experiments have been conducted to demonstrate the superior performance of the proposal.

Recurrent neural network based real-time failure detection of storage devices

Studies have revealed that the failure rates of storage devices can often be as high as fourteen percent. To make matters worse, there are frequently no warning signs for precaution before catastrophic failure of storage devices occurs. A real-time predictive maintenance system that provides an automatic means for predicting when maintenance should be performed to ultimately eliminate unexpected breakdowns needs to be developed. Unlike traditional regression predictive modeling, the failure detection of storage devices is a problem of time series prediction, which adds the complexity of a sequence dependence among the input variables. The proposed LSTM (Long Short-Term Memory) network is a branch of RNN (Recurrent Neural Network) used in deep learning, which presents a very large architecture that can be successfully trained. LSTM is good at extracting patterns in input feature space, where the input data spans over long sequences. With the gated architecture of LSTM, it is capable of learning the context required to make predictions in time series forecasting. It is ideal for generating responses that depend on a time-evolving state; for example detecting the condition of storage devices over time. This paper describes our development of an LSTM (Long short-term memory), a special kind of RNN (Recurrent Neural Network)—based real-time predictive maintenance system (RPMS) built on top of Apache Spark for detecting storage device failure. By streaming real-time data into a RPMS directly from the device itself, the issues can be revealed and addressed early before they cause costly downtime.

Multiple convolutional neural networks for multivariate time series prediction

Multivariate time series prediction, with a profound impact on human social life, has been attracting growing interest in machine learning research. However, the task of time series forecasting is very challenging because it is affected by many complex factors. For example, in predicting traffic and solar power generation, weather can bring great trouble. In particular, for strictly periodic time series, if the periodic information can be extracted from the historical sequence data to the maximum, the accuracy of the prediction will be greatly improved. At present, for time series prediction tasks, the sequence models based on RNN have made great progress. However, the sequence models has difficulty in capturing global information, failing to well highlight the periodic characteristics of the time series. But the this problem can be solved by CNN models. So in this paper, we propose a model called Multiple CNNs to solve the problem of periodic multivariate time series prediction. The working process of Multiple CNNs is analyzing the periodicity of time series, extracting the closeness and the long and short periodic information of the predicted target respectively, and finally integrating the characteristics of the three parts to make the prediction. Moreover, the model is highly flexible, which allows users to freely adjust the cycle span set in the model according to their own data characteristics. Tests on two large real-world datasets, show that our model has a strong advantage over other time series prediction methods.

Short-Term Traffic Flow Prediction Based on Least Square Support Vector Machine with Hybrid Optimization Algorithm

Accurate short-term traffic flow prediction plays an indispensable role for solving traffic congestion. However, the structure of traffic data is nonlinear and complicated. It is a challenge to get high precision. The least square support vector machine (LSSVM) has powerful capabilities for time series and nonlinear regression prediction problems if it can select appropriate parameters. To search the optimal parameters of LSSVM, this paper proposes a hybrid optimization algorithm which combines particle swarm optimization (PSO) with genetic algorithm. The main contributions are twofold: (1) A hybrid optimization method is proposed, which can skip the local optimal pitfall with less learning time by introducing a selection strategy, crossover and mutation operators into PSO; (2) the crossover and mutation operators are controlled by adaptive probability functions. The crossover and mutation probabilities increase when the population fitness is concentrated, and decrease when the fitness is dispersed. It can effectively improve the precision and speed of convergence. The proposed model is verified based on the measured data. The experimental results show that our new model yields better prediction ability and relatively high computational efficiency compared with other related models.

An evolving recurrent interval type-2 intuitionistic fuzzy neural network for online learning and time series prediction

The prediction of time series has both the theoretical value and practical significance in reality. However, since the high nonlinear and noises in the time series, it is still an open problem to tackle with the uncertainties and fuzziness in the forecasting process. In this article, an evolving recurrent interval type-2 intuitionistic fuzzy neural network (eRIT2IFNN) is proposed for time series prediction and regression problems. The eRIT2IFNN employs interval type-2 intuitionistic fuzzy sets to enhance the modeling of uncertainties by intuitionistic evaluation and noise tolerance of the system. In the eRIT2IFNN, the antecedent part of each fuzzy rule is defined using intuitionistic interval type-2 fuzzy sets, and the consequent realizes the Takagi–Sugeno–Kang type fuzzy inference mechanism. In order to utilize the prior knowledge including intuitionistic information, a local internal feedback is established by feeding the rule firing strength of each rule to itself eRIT2IFNN is fully adaptive to the evolving of sequence data by online learning of structure and parameters. A modified density-based clustering is implemented for the structure learning, where both densities and membership degrees are involved to determine the fuzzy rules. Performance of eRIT2IFNN is evaluated using a set of benchmark problems and compared with existing fuzzy inference systems. Moreover, the eRIT2IFNN is tested for identification of dynamics under both noise-free and noisy environments. Finally, a group of practical financial price-tracking problems including high-frequency data of financial future, commodity future and precious metal are used for the evaluation of the proposed inference system.

LSTM based reserve prediction for bank outlets

Reserve allocation is a significant problem faced by commercial banking businesses every day. To satisfy the cash requirement of customers and abate the vault cash pressure, commercial banks need to appropriately allocate reserves for each bank outlet. Excessive reserve would impact the revenue of bank outlets. Low reserves cannot guarantee the successful operation of bank outlets. Considering the reserve requirement is effected by the past cash balance, we deal the reserve allocation problem as a time series prediction problem, and the Long Short Time Memory (LSTM) network is adapted to solve it. In addition, the proposed LSTM prediction model regards date property, which can affect the cash balance, as a primary factor. The experiment results show that our method outperforms some existing traditional methods.

Characterizing and predicting downloads in academic search

Numerous studies have been conducted on the information interaction behavior of search engine users. Few studies have considered information interactions in the domain of academic search. We focus on conversion behavior in this domain. Conversions have been widely studied in the e-commerce domain, e.g., for online shopping and hotel booking, but little is known about conversions in academic search. We start with a description of a unique dataset of a particular type of conversion in academic search, viz. users’ downloads of scientific papers. Then we move to an observational analysis of users’ download actions. We first characterize user actions and show their statistics in sessions. Then we focus on behavioral and topical aspects of downloads, revealing behavioral correlations across download sessions. We discover unique properties that differ from other conversion settings such as online shopping. Using insights gained from these observations, we consider the task of predicting the next download. In particular, we focus on predicting the time until the next download session, and on predicting the number of downloads. We cast these as time series prediction problems and model them using LSTMs. We develop a specialized model built on user segmentations that achieves significant improvements over the state-of-the art.

Kidney-inspired algorithm with reduced functionality treatment for classification and time series prediction

Optimization of an artificial neural network model through the use of optimization algorithms is the common method employed to search for an optimum solution for a broad variety of real-world problems. One such optimization algorithm is the kidney-inspired algorithm (KA) which has recently been proposed in the literature. The algorithm mimics the four processes performed by the kidneys: filtration, reabsorption, secretion, and excretion. However, a human with reduced kidney function needs to undergo additional treatment to improve kidney performance. In the medical field, the glomerular filtration rate (GFR) test is used to check the health of kidneys. The test estimates the amount of blood that passes through the glomeruli each minute. In this paper, we mimic this kidney function test and the GFR result is used to select a suitable step to add to the basic KA process. This novel imitation is designed for both minimization and maximization problems. In the proposed method, depends on GFR test result which is less than 15 or falls between 15 and 60 or is more than 60 a particular action is performed. These additional processes are applied as required with the aim of improving exploration of the search space and increasing the likelihood of the KA finding the optimum solution. The proposed method is tested on test functions and its results are compared with those of the basic KA. Its performance on benchmark classification and time series prediction problems is also examined and compared with that of other available methods in the literature. In addition, the proposed method is applied to a real-world water quality prediction problem. The statistical analysis of all these applications showed that the proposed method had a ability to improve the optimization outcome.

Joint prediction of time series data in inventory management

The problem of time series prediction has been well explored in the community of data mining. However, little research attention has been paid to the case of predicting the movement of a collection of related time series data. In this work, we study the problem of simultaneously predicting multiple time series data using joint predictive models. We observe that in real-world applications, strong relationships between different time-sensitive variables are often held, either explicitly predefined or implicitly covered in nature of the application. Such relationships indicate that the prediction on the trajectory of one given time series could be improved by incorporating the properties of other related time series data into predictive models. The key challenge is to capture the temporal dynamics of these relationships to jointly predict multiple time series. In this research, we propose a predictive model for multiple time series forecasting and apply it to the domain of inventory management. The relationships among multiple time series are modeled as a class of constraints, and in turn, refine the predictions on the corresponding time series. Experimental results on real-world data reveal that the proposed algorithms outperform well-established methods of time series forecasting.

A Novel Torque Distribution Strategy Based on Deep Recurrent Neural Network for Parallel Hybrid Electric Vehicle

In this paper, energy management strategy (EMS) model based on deep recurrent neural network (DRNN) is presented to learn optimal torque distribution for the single-axle parallel hybrid electric vehicle. The model has two distinguishing properties: 1) because the EMS is formulated as a time series prediction problem, taking historical data as input of the EMS model captures the input-and-output dynamic characteristics and enhances the prediction capability and 2) the EMS model based on end-to-end framework directly generates torque distribution results without extracting features of driving cycles and other artificial interference. The extensive simulations are conducted to demonstrate the accuracy and generalization capability of the EMS model in public platform TensorFlow. Comparing with other energy management strategies, our proposed model yields better performance in terms of fuel economy and accuracy. The simulation results show that our proposed EMS model provides a novel way to study the energy management strategy.

A Multi-Patient Data-Driven Approach to Blood Glucose Prediction

Continuous glucose monitoring systems (CGMSs) allow measuring the blood glycaemic value of a diabetic patient at a high sampling rate, producing a considerable amount of data. These data can be effectively used by machine learning techniques to infer future values of the glycaemic concentration, allowing the early prevention of dangerous hyperglycaemic or hypoglycaemic states and better optimization of the diabetic treatment. Most of the approaches in the literature learn a prediction model from the past samples of the same patient, which needs extensive calibrations and limits the usability of the system. In this paper, we investigate the prediction models trained on glucose signals of a large and heterogeneous cohort of patients and then applied to infer future glucose-level values on a completely new patient. To achieve this purpose, we designed and compared two different types of solutions that were proved successful in many time-series prediction problems based respectively, on non-linear autoregressive (NAR) neural network and on long short-term memory (LSTM) networks. These solutions were experimentally compared with three literature approaches, respectively, based on feed-forward neural networks (FNNs), autoregressive (AR) models, and recurrent neural networks (RNN). While the NAR obtained good prediction accuracy only for short-term predictions (i.e., with prediction horizon within 30 min), the LSTM obtained extremely good performance both for short- and long-term glucose-level inference (60 min and more), overcoming all the other methods in terms of correlation between the measured and the predicted glucose signal and in terms of clinical outcome.

On medical application of neural networks trained with various types of data.

Neural networks have garnered attention over the past few years. A neural network is a typical model of machine learning that is used to identify visual patterns. Neural networks are used to solve a wide variety of problems, including image recognition problems and time series prediction problems. In addition, neural networks have been applied to medicine over the past few years. This paper classifies the ways in which neural networks have been applied to medicine based on the type of data used to train those networks. Applications of neural networks to medicine can be categorized two types: automated diagnosis and physician aids. Considering the number of patients per physician, neural networks could be used to diagnose diseases related to the vascular system, heart, brain, spinal column, head, neck, and tumors/cancer in three fields: vascular and interventional radiology, interventional cardiology, and neuroradiology. Lastly, this paper also considers areas of medicine where neural networks can be effectively applied in the future.