Time Series Problems Research Articles

A Deep Learning‐Based Inventory Management and Demand Prediction Optimization Method for Anomaly Detection

The rapid development of emerging technologies such as machine learning and data mining promotes a lot of smart applications, e.g., Internet of things (IoT). The supply chain management and communication are a key research direction in the IoT environment, while the inventory management (IM) has increasingly become a core part of the whole life cycle management process of the supply chain. However, the current situations of a long supply chain life cycle, complex supply chain management, and frequently changing user demands all lead to a sharp rise in logistics and communication cost. Hence, as the core part of the supply chain, effective and predictable IM becomes particularly important. In this way, this work intends to reduce the cost during the life cycle of the supply chain by optimizing the IM process. Specifically, the IM process is firstly formulated as a mathematical model, in which the objective is to jointly minimize the logistic cost and maximize the profit. On this basis, a deep inventory management (DIM) method is proposed to address this model by using the long short‐term memory (LSTM) theory of deep learning (DL). In particular, DIM transforms the time series problem into a supervised learning one and it is trained using the back propagation pattern, such that the training process can be finished efficiently. The experimental results show that the average inventory demand prediction accuracy of DIM exceeds about 80%, which can reduce the inventory cost by about 25% compared with the other state‐of‐the‐art methods and detect the anomaly inventory actions quickly.

Combining Deep Learning and Multiresolution Analysis for Stock Market Forecasting

Due to its complexity, financial time-series forecasting is regarded as one of the most challenging problems. During the past two decades, nonlinear modeling techniques, such as artificial neural networks, were commonly employed to solve a variety of time-series problems. Recently, however, deep neural network has been found to be more efficient than those in many application domains. In this article, we propose a model based on deep neural networks that improves the forecasting of stock prices. We investigate the impact of combining deep learning techniques with multiresolution analysis to improve the forecasting accuracy. Our proposed model is based on an empirical wavelet transform which is shown to outperform traditional stationary wavelet transform in capturing price fluctuations at different time scales. The proposed model is demonstrated to be substantially more effective than other models when evaluated on the S&P500 stock index and Mackey-Glass time series.

Forecasting daily stock trend using multi-filter feature selection and deep learning

Stock market forecasting has attracted significant attention mainly due to the potential monetary benefits. Predicting these markets is a challenging task due to numerous interrelated factors, and needs a complete and efficient feature selection process to identify the most informative factors. As a time series problem, stock price movements are also dependent on movements on its previous trading days. Feature selection techniques have been widely applied in stock forecasting, but existing approaches usually use a single feature selection technique, which may overlook some important assumptions about the underlying regression function linking the input and output variables. In this study, we combine features selected by multiple feature selection techniques to generate an optimal feature subset and then use a deep generative model to predict future price movements. First, we compute an extended set of forty-four technical indicators from daily stock data of eighty-eight stocks and then compute their importance by independently training logistic regression model, support vector machine and random forests. Based on a prespecified threshold, the lowest ranked features are dropped and the rest are grouped into clusters. The variable importance measure is reused to select the most important feature from each cluster to generate the final subset. The input is then fed to a deep generative model comprising of a market signal extractor and an attention mechanism. The market signal extractor recurrently decodes market movement from the latent variables to deal with stochastic nature of the stock data and the attention mechanism discriminates between predictive dependencies of different temporal auxiliary outputs. The results demonstrate that combining features selected by multiple feature selection approaches and using them as input into a deep generative model outperforms state-of-the-art approaches.

Extreme learning machine ensemble model for time series forecasting boosted by PSO: Application to an electric consumption problem

Ensemble Model is a tool that has attracted attention due to its capability to improve the outcome performance of emerging techniques to solve the modelling and classifying problem. However, the feasibility of applying intelligent algorithms to build the Ensemble Model presents a challenge of its own. In this work, an Extreme Learning Machine ensemble is applied to the Time Series modelling problem. We develop a thorough study of the models that will be candidates to compose the ensemble, obtaining statistical results of optimal topologies to solve each Time Series problem. The proposed method for the ensemble is the weighted averaging method, where the parameters (weights) are tuned with the Particle Swarm Optimization algorithm. Lastly, the ensemble is tested first in the well known Santa Fe Time Series Competition benchmark. Given the obtained satisfactory results, the ensemble is secondly tested in a real problem of Spain’s electric consumption forecasting. It is demonstrated that the PSO is a suitable algorithm to optimize Extreme Learning Machine ensemble and that the proposed strategy can obtain good results in both Time Series problems.

Leveraging spatial information to forecast nonlinear ecological dynamics

Abstract There has been a recent demand for forecasting in ecology, particularly in the field of ecosystem management. Empirical dynamic modelling (EDM), an equation‐free nonlinear forecasting method, is receiving growing attention, but it requires long time series to produce accurate predictions. Though most ecological time series are short, spatial replicates are often available. Here we explore how utilizing available spatial data can improve our ability to forecast ecological dynamics. There are several ways to incorporate spatial information into EDM and not all have been applied in ecology. We compare spatial EDM approaches used in ecology and physics and introduce a flexible Bayesian model that makes use of prior movement information. We test these methods on simulated data generated with three population dynamics models with varying levels of complexity, time series length, spatial symmetry and heterogeneity. Adding spatial data generally improves accuracy, though the best method depends on the spatial process. We applied the methods to empirical fisheries data, highlighting the complexity of real population dynamics. Leveraging spatial data is an effective way to overcome the problem of short ecological time series. Since the best forecasting method depends on the underlying dynamics, we suggest that users apply several in concert and that this may be useful in identifying spatial heterogeneity in dynamics.

Analysis and Prediction of Alerts in Perimeter Intrusion Detection System

Perimeter surveillance systems play an important role in the safety and security of the armed forces. These systems tend to generate alerts in advent of anomalous situations, which require human intervention. The challenge is the generation of false alerts or alert flooding which makes these systems inefficient. In this paper, we focus on short-term as well as long-term prediction of alerts in the perimeter intrusion detection system. We have explored the dependent and independent aspects of the alert data generated over a period of time. Short-term prediction is realized by exploiting the independent aspect of data by narrowing it down to a time-series problem. Time-series analysis is performed by extracting the statistical information from the historical alert data. A dual-stage approach is employed for analyzing the time-series data and support vector regression is used as the regression technique. It is helpful to predict the number of alerts for the nth hour. Additionally, to understand the dependent aspect, we have investigated that the deployment environment has an impact on the alerts generated. Long-term predictions are made by extracting the features based on the deployment environment and training the dataset using different regression models. Also, we have compared the predicted and expected alerts to recognize anomalous behaviour. This will help in realizing the situations of alert flooding over the potential threat.

Neural Networks as Classification Mechanisms of Complex Human Activities

Within this paper, we present two neural nets for view-independent complex human activity recognition (HAR) from video frames. For our study here, we reduce the number of frames produced by a video sequence given that we can identify activities from a sparsely sampled sequence of body poses, and, at the same time, we are able to reduce the processing complexity and response while hardly affecting the accuracy, precision, and recall. To do so, we use a formal framework to ensure the quality of data collection and data preprocessing. We utilize neural networks for the classification of single and complex body activities. More specifically, we consider the sequence of body poses as a time-series problem given that they can provide state-of-the-art results on challenging recognition tasks with little data engineering. Deep Learning in the form of Convolutional Neural Network (CNN), Long Short-Term Neural Network (LSTM), and a one-dimensional Convolutional Neural Network Long Short-Term Memory model (CNN-LSTM) are used as benchmarks to classify the activity.

Trend-based time series data clustering for wind speed forecasting

Wind forecasting is a time series problem, can aide in estimating the annual energy production of potential wind farms. Seasonality and trend are the two significant components that characterize the wind time series data. Variability in trend and seasonal component affects the performance of most of the forecasting methods. Therefore, to simplify the wind forecasting technique, generally, nonlinear seasonal and trend components are eliminated from wind time series data. Accuracy depends on the application function that is applicable to eliminate the trend and seasonality. In this article, a hybrid approach for time series forecasting has been proposed. A clustering technique has been developed, which finds the clusters of time series data showing identical trend components. After finding the proper clusters of similar trend components, statistical methods, namely, autoregressive integrated moving average and generalized autoregressive score techniques, are applied to the individual cluster. In the end, resulting components are aggregated. The experiment shows that the cluster-based forecasting technique gives better performance as compared with existing statistical models.

Occurrence prediction of cotton pests and diseases by bidirectional long short-term memory networks with climate and atmosphere circulation

The occurrence of crop pests and diseases always affects the development of agriculture seriously, while pest meteorology showed that climate is important in affecting the occurrence. Recently, recurrent neural network (RNN) has been broadly applied in various fields, which was designed for modeling sequential data and has been testified to be quite efficient in time series problem. This paper proposes to use bi-directional RNN with long short-term memory (LSTM) units for predicting the occurrence of cotton pests and diseases with climate factors. First, the problem of occurrence prediction of pests and diseases is formulated as time series prediction. Then the bi-directional LSTM network (Bi-LSTM) is adopted to solve the problem, which can capture long-term dependencies on the past and future contexts of sequential data. Experimental results showed that Bi-LSTM shows good performance on the occurrence prediction of pests and diseases in cotton fields, and yields an Area Under the Curve (AUC) of 0.95. This work further verified that climate indeed have strong impact on the occurrence of pests and diseases, and circulation parameters also have certain influence.

AN USER INTENTION MINING MODEL BASED ON FRACTAL TIME SERIES PATTERN

Users use the network more and more frequently, and more and more data is published on the network. Therefore, how to find, organize, and use the useful information behind these massive data through effective means, and analyze user intentions is a huge challenge. There are many time series problems in user intentions. Time series have complex characteristics such as randomness and multi-scale variability. Effectively identifying the inherent laws and objective phenomena contained in time series is the purpose of analyzing and processing time series data. Fractal theory provides a new way to analyze time series, and obtains the characteristics and rules of time series from a new perspective. Therefore, this paper introduces the fractal theory to analyze the time series problem, and proposes an improved G-P algorithm to realize the prediction and mining of user intentions. First, the method of array storage instead of repeated calculations is used to improve the method of saturated correlation dimension. Second, the Hurst exponent of the time series is obtained by the variable scale range analysis method. Finally, a fractal model for predicting user intent in short time series is established using the accumulation and transformation method. The experimental results show that the use of fractal theory can effectively describe the relevant characteristics of time series, the development trend of user intentions can be mined from big data, and the prediction model for short time series can be established to achieve information mining of user intentions.

Improving maritime traffic emission estimations on missing data with CRBMs

Maritime traffic emissions are a major concern to governments as they heavily impact the Air Quality in coastal cities. Ships use the Automatic Identification System (AIS) to continuously report position and speed among other features, and therefore this data is suitable to be used to estimate emissions, if it is combined with engine data. However, important ship features are often inaccurate or missing. State-of-the-art complex systems, like CALIOPE at the Barcelona Supercomputing Center, are used to model Air Quality. These systems can benefit from AIS based emission models as they are very precise in positioning the pollution. Unfortunately, these models are sensitive to missing or corrupted data, and therefore they need data curation techniques to significantly improve the estimation accuracy. In this work, we propose a methodology for treating ship data using Conditional Restricted Boltzmann Machines (CRBMs) plus machine learning methods to improve the quality of data passed to emission models that can also be applied to other GPS and time-series problems. Results show that we can improve the default methods proposed to cover missing data. In our results, we observed that using our method the models boosted their accuracy to detect otherwise undetectable emissions. In particular, we used a real data-set of AIS data, provided by the Spanish Port Authority, to estimate that thanks to our method, the model was able to detect 45% of additional emissions, representing 152 tonnes of pollutants per week in Barcelona and propose new features that may enhance emission modeling.

Efficient segmentation-based methods for anomaly detection in static and streaming time series under dynamic time warping

The problem of time series anomaly detection has attracted a lot of attention due to its usefulness in various application domains. However, most of the methods proposed so far used Euclidean distance to deal with this problem. Dynamic Time Warping (DTW) distance is more suitable than Euclidean distance because of its capability in shape-based similarity checking in many practical fields, for example those with multimedia data. In this paper, we propose two efficient anomaly detection methods, EP-Leader-DTW and SEP-Leader-DTW, for static and streaming time series under DTW, respectively. Our methods are based on time series segmentation, subsequence clustering, and anomaly scoring. For segmentation, the major extrema method is used to obtain subsequences. For clustering, we apply Leader algorithm to cluster the subsequences along with a lower bounding technique to accelerate DTW distance computation. Experimental results on several benchmark time series datasets reveal that our method for anomaly detection in static time series under DTW can perform very fast and accurately on large time series datasets. For streaming time series, our method can meet the instantaneous requirement with high accuracy. As a result, our anomaly detection methods are applicable to both static and streaming time series in practice.

A flight maneuver recognition method based on multi-strategy affine canonical time warping

Maneuver recognition for unmanned combat air vehicle (UCAV) is a necessary technique for autonomous air combat. As a spatiotemporal alignment problem of multidimensional time series, the flight maneuver recognition is solved by a novel alignment measure, multi-strategy affine canonical time warping approach (MACTW) and its derivative form, which are extensions of affine canonical time warping (ACTW). MACTW makes several contributions: (1) it proposes multi-strategy success-history based adaptive differential evolution algorithm with linear population size reduction (MLSHADE) to accelerate the search of warping path of dynamic time warping (DTW); (2) it introduces affine strategy to address offset and scaling of canonical time warping (CTW), which is a combination of DTW and canonical correlation analysis (CCA); and (3) it extends ACTW based on MLSHADE to align multidimensional time series In addition, MLSHADE is a novel optimization technique that employs weighted mutation, inferior solution search, and eigen Gaussian walk strategies to improve the optimization efficiency. The experimental results on the CEC 2018 test suite illustrate the superior benefits of MLSHADE. UCAV flight maneuver recognition system which includes segmentation, preprocessing and recognition modules is modeled. The experimental results on UCR datasets and UCAV maneuver datasets including action units and long maneuver datasets illustrate the superiority of MACTW and its derivative form compared with other state-of-the-art alignment measures.

Detecting changes in the second moment structure of high-dimensional sensor-type data in a K-sample setting

The K sample problem for high-dimensional vector time series is studied, especially focusing on sensor data streams, in order to analyze the second moment structure and detect changes across samples and/or across variables cumulated sum (CUSUM) statistics of bilinear forms of the sample covariance matrix. In this model, K independent vector time series are observed over a time span which may correspond to K sensors (locations) yielding d-dimensional data as well as K locations where d sensors emit univariate data. Unequal sample sizes are considered as arising when the sampling rate of the sensors differs. We provide large-sample approximations and two related change point statistics, a sum of squares and a pooled variance statistic. The resulting procedures are investigated by simulations and illustrated by analyzing a real data set.

Bayesian Neuro-Fuzzy Inference System for Temporal Dependence Estimation

When it comes to time-series forecasting, it is crucial to learn the intricate temporal relationship between past and future, and historical information is of paramount importance for this purpose. Traditional neuro-fuzzy systems generally resort to an empirical (offline) method to determine the number of past instances (i.e., historical information) required for a particular model, hence often unsuitable in an online time-series scenario. In this article, we propose a Bayesian neuro-fuzzy inference system (BaNFIS), where the temporal dependence on past instances is estimated with an online Bayesian probabilistic mechanism, and the uncertainty associated with real-world data is handled by the fuzzy inference system. The BaNFIS retains historical information only as per necessity and employs it in two ways: globally or locally. Moreover, an online learning method is employed here to update the BaNFIS parameters. Hence, the BaNFIS is able to capture both the system dynamics and uncertainty efficiently in an online manner. Three real-world time-series problems are employed here to evaluate the online performance of the BaNFIS compared to seven state-of-the-art neuro-fuzzy methods both under standard train–test and prequential test–train protocols. Numerical results clearly indicate that the BaNFIS provides a statistically improved prediction performance than its peers in terms of accuracy.

Symbiotic organisms search-optimized deep learning technique for mapping construction cash flow considering complexity of project

Abstract Accurate construction cash flow forecasting is very important in successfully managing cost during execution of building projects. Despite many research efforts, it still remains a difficult issue in attaining an accurate forecast model of cash flows due to the risk factors and characteristic of the project. Additionally, cash flow of the construction projects is strongly impacted by sequence and non-sequence factors. Hence, this study proposed a novel artificial intelligence(AI)-based inference model, named symbiotic organisms search-optimized neural network-long short-term memory (SOS-NN-LSTM), which employs symbiotic organisms search (SOS) algorithm to obtain the suitable hyperparameters of the neural network (NN) and long short-term memory (LSTM) for establishing a robust hybridization model. In the proposed model, the LSTM technique addresses time series problem with considering the complexity of projects while the NN technique aims at tackling non-sequence factors. The experimental results on 13 construction projects have supported the SOS-NN-LSTM as the best model in forecasting the cash flow by achieving the greatest values of (2.55%), MAPE (5.71%), MAE (2.07%), and R2 (0.983). The statistical result further reveals that accuracy of cash flow forecasting can be improved at least 13.4% and 12.0% in terms of RMSE and MAE, respectively, in comparison with other comparative AI-based inference models. The SOS-NN-LSTM model is thus a useful tool to help managers forecast and control cash flow of construction projects.

A support vector regression model hybridized with chaotic krill herd algorithm and empirical mode decomposition for regression task

This work presents a hybrid model that combines support vector regression (SVR), empirical mode decomposition (EMD), the krill herd (KH) algorithm and a chaotic mapping function. EMD is used to decompose input time series data into components with several intrinsic mode functions (IMFs) and one residual, to capture the trends in the input data. SVR is used to forecast separately IMFs and the residual owing to its effectiveness in solving nonlinear regression and time series problems. The KH algorithm is used to select the parameters in the SVR models. The Tent chaotic mapping function is hybridized with the KH algorithm to prevent premature convergence and increase the accuracy of the whole model. Two real-world datasets from the New South Wales (NSW, Australia) market and the New York Independent System Operator (NYISO, USA) are used to demonstrate the performance of the proposed EMD-SVRCKH model. The experimental results reveal that the proposed model provides competitive advantages over other models and offers greater forecasting accuracy.

Data driven governing equations approximations using attention based multistep neural networks

During the past decade, deep learning has represented the biggest research trend in the field of machine learning, which provides new powerful tools to interrogate high dimensional time series data in a way that has not been possible before. With the recent success in natural language processing, one would expect widespread adaptation to problems like time series forecasting and classification. After all, both involve processing sequential data. However, to this point, research on their adaptation to time series problems has remained limited. Recently, a multi-step time-stepping scheme without the need of direct access to temporal gradients has been proposed, which can accurately identify nonlinear dynamical systems from time series data. In this paper, we combined an attention mechanism with a deep model in a multi-step time-stepping scheme to perform nonlinear system identification and forecasting tasks. To our knowledge, this is the first paper to use attention based models to deal with nonlinear system identification and forecasting problems. The attention weights on rows select those variables that are helpful for forecasting, which can enhance the information across multiple time steps to capture temporal information. The experiment results indicate that the attention based model in a multi-step time-stepping scheme has better identification and prediction performance for nonlinear time series identification and forecasting problems.

A Continuous-Decision Virtual Network Embedding Scheme Relying on Reinforcement Learning

Network Virtualization (NV) techniques allow multiple virtual network requests to beneficially share resources on the same substrate network, such as node computational resources and link bandwidth. As the most famous family member of NV techniques, virtual network embedding is capable of efficiently allocating the limited network resources to the users on the same substrate network. However, traditional heuristic virtual network embedding algorithms generally follow a static operating mechanism, which cannot adapt well to the dynamic network structures and environments, resulting in inferior nodes ranking and embedding strategies. Some reinforcement learning aided embedding algorithms have been conceived to dynamically update the decision-making strategies, while the node embedding of the same request is discretized and its continuity is ignored. To address this problem, a Continuous-Decision virtual network embedding scheme relying on Reinforcement Learning (CDRL) is proposed in our paper, which regards the node embedding of the same request as a time-series problem formulated by the classic seq2seq model. Moreover, two traditional heuristic embedding algorithms as well as the classic reinforcement learning aided embedding algorithm are used for benchmarking our prpposed CDRL algorithm. Finally, simulation results show that our proposed algorithm is superior to the other three algorithms in terms of long-term average revenue, revenue to cost and acceptance ratio.

Airborne particle pollution predictive model using Gated Recurrent Unit (GRU) deep neural networks

Developments in deep learning for time-series problems have shown promising results for data prediction. Particulate Matter equal or smaller than 10 μm (PM10) have increased importance in the research field due to the negative impact in the respiratory system. PM10 particles show non-linear behavior, hence it is not an easy task to implement techniques to predict subsequent concentration of the particles in the atmosphere. This paper presents a forecasting model using gated Recurrent unit (GRU) and Long-Short Term Memory (LSTM) networks, which are types of a deep recurrent neural network (RNN). The predicted results of PM10 are presented using data of Mexico City as a case study, showing that this type of deep network is feasible for predicting the non-linearities of this type of data. Several experiments were carried out for 12, 24, 48, and 120 h prediction, showing that this method may be applied to accurately forecast the behavior of PM10.