Accurate Time Series Research Articles

An Enterprise Time Series Forecasting System for Cloud Applications Using Transfer Learning

The main purpose of an application performance monitoring/management (APM) software is to ensure the highest availability, efficiency and security of applications. An APM software accomplishes the main goals through automation, measurements, analysis and diagnostics. Gartner specifies the three crucial capabilities of APM softwares. The first is an end-user experience monitoring for revealing the interactions of users with application and infrastructure components. The second is application discovery, diagnostics and tracing. The third key component is machine learning (ML) and artificial intelligence (AI) powered data analytics for predictions, anomaly detection, event correlations and root cause analysis. Time series metrics, logs and traces are the three pillars of observability and the valuable source of information for IT operations. Accurate, scalable and robust time series forecasting and anomaly detection are the requested capabilities of the analytics. Approaches based on neural networks (NN) and deep learning gain an increasing popularity due to their flexibility and ability to tackle complex nonlinear problems. However, some of the disadvantages of NN-based models for distributed cloud applications mitigate expectations and require specific approaches. We demonstrate how NN-models, pretrained on a global time series database, can be applied to customer specific data using transfer learning. In general, NN-models adequately operate only on stationary time series. Application to nonstationary time series requires multilayer data processing including hypothesis testing for data categorization, category specific transformations into stationary data, forecasting and backward transformations. We present the mathematical background of this approach and discuss experimental results based on implementation for Wavefront by VMware (an APM software) while monitoring real customer cloud environments.

ELIS++: a shapelet learning approach for accurate and efficient time series classification

In recent years, time series classification with shapelets, due to the high accuracy and good interpretability, has attracted considerable interests. These approaches extract or learn shapelets from the training time series. Although they can achieve higher accuracy than other approaches, there still confront some challenges. First, they may suffer from low accuracy in the case of small training dataset. Second, they must manually set some parameters, like the number of shapelets and the length of each shapelet beforehand, and some hyper-parameters, like learning rate and regulation weight, which are difficult to set without prior knowledge. Third, extracting or learning shapelets incurs a huge computation cost, due to the huge search space. In this paper, we extend our previous shapelet learning approach ELIS to ELIS++. To improve the accuracy on the small training dataset, we propose a data augmentation approach. To learn the higher quality shapelets, based on the PAA shapelet candidates search technique proposed in ELIS, ELIS++ first propose a novel entropy-based approach shapelet candidate selection mechanism to discover shapelet candidates, and then applies the logistic regression model to adjust shapelets.To avoid setting other parameters manually, we propose a Bayesian Optimization based approach. Moreover, two techniques are proposed to improve the efficiency, coarse-grained shapelet adjustment and SIMD-based parallel computation. We conduct extensive experiments on 35 UCR datasets, and results verify the effectiveness and efficiency of ELIS++.

Improving time series forecasting using LSTM and attention models

Accurate time series forecasting has been recognized as an essential task in many application domains. Real-world time series data often consist of non-linear patterns with complexities that prevent conventional forecasting techniques from accurate predictions. To forecast a given time series accurately, a hybrid model based on two deep learning methods, i.e., long short-term memory (LSTM) and multi-head attention is proposed in this study. The proposed method leverages the two learned representations from these techniques. The performance of this method is also compared with some standard time series forecasting techniques as well as some hybrid cases proposed in the related literature using 16 datasets. Moreover, the individual models based on LSTM and multi-head attention are implemented to perform a comprehensive evaluation. The results of experiments in this study indicate that the proposed model outperforms all benchmarking methods in most datasets in terms of symmetric mean absolute percentage error (SMAPE). It yields the best average rank (AR) among the utilized methods. Besides, the results reveal that model based on multi-head attention is the second-best method with regard to AR, which demonstrates the predictive power of attention mechanism in time series forecasting.

Temporal Consolidation Strategy for Ground-Based Image Displacement Time Series: Application to Glacier Monitoring

In this paper, we present a method for combining image-based displacements to build time series. This method takes advantage of the redundancy of these displacements which comes from multiple possible combinations between images. The proposed method combines common master series having a different master date, in order to construct a single time series of relative displacements. We were able to test this method on displacements computed using images from a stereo time lapse device recording images of the Argentire glacier during the summer and fall 2019. Our method was compared to two other displacement aggregation strategies: a simple common master approach and the classical inversion method. In order to perform this comparison, displacement data from 4 permanent GPS are used. The results show that our method provides more accurate time series of relative displacements than those obtained with the other methods. The higher accuracy of the proposed method makes the detection of seasonal variations in glacier velocity possible.

A survey on long short-term memory networks for time series prediction

Recurrent neural networks and exceedingly Long short-term memory (LSTM) have been investigated intensively in recent years due to their ability to model and predict nonlinear time-variant system dynamics. The present paper delivers a comprehensive overview of existing LSTM cell derivatives and network architectures for time series prediction. A categorization in LSTM with optimized cell state representations and LSTM with interacting cell states is proposed. The investigated approaches are evaluated against defined requirements being relevant for an accurate time series prediction. These include short-term and long-term memory behavior, the ability for multimodal and multi-step ahead predictions and the according error propagation. Sequence-to-sequence networks with partially conditioning outperform the other approaches, such as bidirectional or associative networks, and are best suited to fulfill the requirements.

Improving Stage–Discharge Relation in The Mekong River Estuary by Remotely Sensed Long-Period Ocean Tides

Ocean tidal backwater reshapes the stage–discharge relation in the fluvial-to-marine transition zone at estuaries, rendering the cautious use of these data for hydrological studies. While a qualitative explanation is traditionally provided by examining a scatter plot of water discharge against water level, a quantitative assessment of long-period ocean tidal effect on the stage–discharge relation has been rarely investigated. This study analyzes the relationship among water level, water discharge, and ocean tidal height via their standardized forms in the Mekong Delta. We found that semiannual and annual components of ocean tides contribute significantly to the discrepancy between standardized water level and standardized water discharge time series. This reveals that the long-period ocean tides are the significant factors influencing the stage–discharge relation in the river delta, implying a potential of improving the relation as long as proper long-period ocean tidal components are taken into consideration. By isolating the short-period signals (i.e., less than 15 days) from land surface hydrology and ocean tides, better consistent stage–discharge relations are obtained, in terms of improving the Pearson correlation coefficient (PCC) from ~0.4 to ~0.8 and from ~0.6 to ~0.9 for the stations closest to the estuary and at the Mekong Delta entrance, respectively. By incorporating the long-period ocean tidal height time series generated from a remotely sensed global ocean tide model into the stage–discharge relation, further refined stage–discharge relations are obtained with the PCC higher than 0.9 for all employed stations, suggesting the improvement of daily averaged water level and water discharge while ignoring the short-period intratidal variability. The remotely sensed global ocean tide model, OSU12, which contains annual and semiannual ocean tide components, is capable of generating accurate tidal height time series necessary for the partial recovery of the stage–discharge relation.

Mapping National-Scale Croplands in Pakistan by Combining Dynamic Time Warping Algorithm and Density-Based Spatial Clustering of Applications with Noise

Croplands are commonly mapped using time series of remotely sensed images. The dynamic time warping (DTW) algorithm is an effective method for realizing this. However, DTW algorithm faces the challenge of capturing complete and accurate representative cropland time series on a national scale, especially in Asian countries where climatic and topographic conditions, cropland types, and crop growth patterns vary significantly. This study proposes an automatic cropland extraction method based on the DTW algorithm and density-based spatial clustering of applications with noise (DBSCAN), hereinafter referred to as ACE-DTW, to map croplands in Pakistan in 2015. First, 422 frames of multispectral Landsat-8 satellite images were selected from the Google Earth Engine to construct monthly normalized difference vegetation index (NDVI) time series. Next, a total of 2409 training samples of six land cover types were generated randomly and explained visually using high-resolution remotely sensed images. Then, a multi-layer DBSCAN was used to classify NDVI time series of training samples into different categories automatically based on their pairwise DTW distances, and the mean NDVI time series of each category was used as the standard time series to represent the characteristics of that category. These standard time series attempted to represent cropland information and maximally distinguished croplands from other possible interference land cover types. Finally, image pixels were classified as cropland or non-cropland based on their DTW distances to the standard time series of the six land cover types. The overall cropland extraction accuracy of ACE-DTW was 89.7%, which exceeded those of other supervised classifiers (classification and regression trees: 78.2%; support vector machines: 78.8%) and existing global cropland datasets (Finer Resolution Observation and Monitoring of Global Land Cover: 87.1%; Global Food Security Support Analysis Data: 83.1%). Further, ACE-DTW could produce relatively complete time series of variable cropland types, and thereby provide a significant advantage in mountain regions with small, fragmented croplands and plain regions with large, high-density patches of croplands.

Identifying and Quantifying Urban Polycentric Development in China From DMSP-OLS Data and Urban Land Data Sets

This study attempted to identify and quantify the morphology of intercity urban polycentric development (UPD) from the Defense Meteorological Satellite Program’s Operational Linescan System (DMSP-OLS) data and urban land (UL) data sets in China at the provincial level. The spatiotemporal change and impact factors of UPD from 2000 to 2012 were also evaluated. The accuracy verification results indicated that the UPD could effectively and accurately identify and evaluate from the DMSP-OLS data and UL data sets in China. China’s urban structure presented a UPD trend from 2000 to 2012 and showed a pattern of high values in the eastern region and low values in the western region. In addition, the gross domestic product was proven to be a significant factor and had an inverted U-shaped impact on the UPD. The study can provide accurate time-series UPD data sets for decision-makers to evaluate the spatiotemporal change and driving mechanism of the intercity urban structure in China at the provincial level.

Air Transportation Direct Share Time Series Forecasting: A Hybrid Model

In modern air transportation, the direct share is the ratio of direct passengers to total passengers on a directional origin and destination (O&D) pair. The forecasting of direct share time series on the O&D level, as part of the detailed demand forecasting, plays a fundamental role in air transportation planning and development. An accurate forecasting of the O&D direct share time series can benefit the air transportation planners, airlines, and airports in multiple ways. Based on the previous analysis, the direct share time series is O&D specific. This research focuses on developing accurate direct share time series forecasting models on O&D markets with different characteristics. Both classical time series models and supervised learning regression models are investigated carefully. A novel hybrid model that combines time series concept and machine learning modeling techniques is proposed, which can provide more accurate forecasting performance and valuable insights into the O&D markets. To automatically select the forecasting model for each O&D pair, a general modeling framework is proposed for direct share time series forecasting. Based on the forecasting performance comparison, the modeling framework can provide promising direct share time series forecasting, which is a reliable replacement for the model used in the Federal Aviation Administration Terminal Area Forecast.

A novel ensemble deep learning model with dynamic error correction and multi-objective ensemble pruning for time series forecasting

In the past decade, deep learning models have shown to be promising tools for time series forecasting. However, owing to significant differences in the volatility characteristics among different types of time series data, it is difficult for an individual deep learning model to maintain robust forecasting performance. In this study, a novel ensemble deep learning model is proposed to achieve accurate and stable time series forecasting. First, a boosting deep learning method based on extended AdaBoost algorithm is proposed for generating various basic predictors. These basic predictors are further enhanced through a new dynamic error correction method. A stacking-based ensemble method that employs kernel ridge regression as the meta-predictor is then used to combine the basic predictors to produce the ultimate forecasting results. To increase forecasting accuracy and stability, an enhanced multi-population non-dominated sorting genetic algorithm-II is proposed for ensemble pruning. Finally, the forecasting performance of the proposed model is verified through the use of three different types of real-world time series data (i.e., PM2.5 concentration, wind speed, and electricity price). The experimental results showed that the proposed model is superior to other baseline models in dealing with time series forecasting tasks.

A Time Series Forecasting for the Cumulative Confirmed and Critical Cases of the Covid-19 Pandemic in Saudi Arabia using Autoregressive Integrated Moving Average (ARIMA) Model

Reviews at present, different machine learning techniques and algorithms have been applied for predicting significant factors of the Coronavirus Disease-2019 (COVID-19) such as the outbreak and diagnosis. In this study, the most accurate time series forecasting model, namely, the Autoregressive Integrated Moving Average (ARIMA) model is used to forecast the expected cumulative number of confirmed and critical cases in Saudi Arabia for the upcoming months. Additionally, the dataset is collected from the King Abdullah Petroleum Studies and Research Centre (KAPSARC). Acquiring the number of expected cases within a short period is considered crucial as it provides an important knowledge that can be applied by the health sector in containing the COVID-19 pandemic and forming the proper precautions and strategies that are concerned on the public health system. The main finding of this research is that the number of cumulative confirmed cases is expected to increase at a high rate in the upcoming two months, while the number of critical cases is forecasted to increase at a smaller rate compared to the total number of cases. To evaluate the performance of the adopted model, different statistical matrices as the R Squared, Mean Squarer Error, Root Mean Square Error and Mean Absolute Error are used in this research. It is found to be proven from the findings that the proposed model generates an accurate prediction of the expected number of cumulative confirmed and critical cases in the upcoming months.

ROCKET: exceptionally fast and accurate time series classification using random convolutional kernels

Most methods for time series classification that attain state-of-the-art accuracy have high computational complexity, requiring significant training time even for smaller datasets, and are intractable for larger datasets. Additionally, many existing methods focus on a single type of feature such as shape or frequency. Building on the recent success of convolutional neural networks for time series classification, we show that simple linear classifiers using random convolutional kernels achieve state-of-the-art accuracy with a fraction of the computational expense of existing methods. Using this method, it is possible to train and test a classifier on all 85 ‘bake off’ datasets in the UCR archive in $$<\,2\,\hbox {h}$$ , and it is possible to train a classifier on a large dataset of more than one million time series in approximately 1 h.

Intelligent fault prediction of rolling bearing based on gate recurrent unit and hybrid autoencoder

Accurate fault prediction of rolling bearing can predict the operation condition in advance, which is an important means to ensure the safety and reliability of rotating machinery. Aimed at the data processing of rolling bearing vibration signal with multi-fault and long time series, an intelligent fault prediction model based on gate recurrent unit and hybrid autoencoder is proposed in this paper. Firstly, vibration signals of multi-faults are brought into multi-layer gate recurrent unit model for multi-step and multi-variable time series prediction. Secondly, variational autoencoder is used for data augmentation of fault samples. Thirdly, the augmented fault samples are brought into stacked denoising autoencoder for noise reduction and fault prediction. Finally, fault prediction results of rolling bearing can be achieved on the basis of gate recurrent unit and hybrid autoencoder of variational autoencoder and stacked denoising autoencoder. The bearing datasets of Case Western Reserve University are used to verify the effectiveness of the proposed method. Comparative experiment results show that the proposed fault prediction model has more accurate time series prediction result and higher fault prediction accuracy than other deep learning models. With 98.68% accuracy of fault prediction, the proposed fault prediction model can be taken as an effective tool for intelligent predictive maintenance of rolling bearing.

Optimal Input Variables Disposition of Artificial Neural Networks Models for Enhancing Time Series Forecasting Accuracy

ABSTRACT Artificial Neural Networks (ANNs) models play an increasingly significant role in accurate time series prediction tools. However, an accurate time series forecasting using ANN requires an optimal model. Hence, great forecasting methods have been developed from optimized ANN models. Most of them focus more on input variables selection and preprocessing, topologies selection, optimum configuration and its associated parameters regardless of their input variables disposition. This paper provides an investigation of the effects of input variables disposition on ANNs models on training and forecasting performances. After investigation, a new ANNs optimization approach is proposed, consisting of finding optimal input variables disposition from the possible combinations. Therefore, a modified Back-Propagation neural networks training algorithm is presented in this paper. This proposed approach is applied to optimize the feed-forward and recurrent neural networks architectures; both built using traditional techniques, and pursuing to forecast the wind speed. Furthermore, the proposed approach is tested in a collaborative optimization method with single-objective optimization technique. Thus, Genetic Algorithm Back-Propagation neural networks aim to improve the forecasting accuracy relative to traditional methods was proposed. The experiment results demonstrate the requirement to take into consideration the input variables disposition to build a more optimal ANN model. They reveal that each proposed model is superior to its old considered model in terms of forecasting accuracy and thus show that the proposed optimization approach can be useful for time series forecasting accuracy improvement.

TEASER: early and accurate time series classification

Early time series classification (eTSC) is the problem of classifying a time series after as few measurements as possible with the highest possible accuracy. The most critical issue of any eTSC method is to decide when enough data of a time series has been seen to take a decision: Waiting for more data points usually makes the classification problem easier but delays the time in which a classification is made; in contrast, earlier classification has to cope with less input data, often leading to inferior accuracy. The state-of-the-art eTSC methods compute a fixed optimal decision time assuming that every times series has the same defined start time (like turning on a machine). However, in many real-life applications measurements start at arbitrary times (like measuring heartbeats of a patient), implying that the best time for taking a decision varies widely between time series. We present TEASER, a novel algorithm that models eTSC as a two-tier classification problem: In the first tier, a classifier periodically assesses the incoming time series to compute class probabilities. However, these class probabilities are only used as output label if a second-tier classifier decides that the predicted label is reliable enough, which can happen after a different number of measurements. In an evaluation using 45 benchmark datasets, TEASER is two to three times earlier at predictions than its competitors while reaching the same or an even higher classification accuracy. We further show TEASER’s superior performance using real-life use cases, namely energy monitoring, and gait detection.

An improved post-processing technique for automatic precipitation gauge time series

Abstract. The unconditioned data retrieved from accumulating automated weighing precipitation gauges are inherently noisy due to the sensitivity of the instruments to mechanical and electrical interference. This noise, combined with diurnal oscillations and signal drift from evaporation of the bucket contents, can make accurate precipitation estimates challenging. Relative to rainfall, errors in the measurement of solid precipitation are exacerbated because the lower accumulation rates are more impacted by measurement noise. Precipitation gauge measurement post-processing techniques are used by Environment and Climate Change Canada in research and operational monitoring to filter cumulative precipitation time series derived from high-frequency, bucket-weight measurements. Four techniques are described and tested here: (1) the operational 15 min filter (O15), (2) the neutral aggregating filter (NAF), (3) the supervised neutral aggregating filter (NAF-S), and (4) the segmented neutral aggregating filter (NAF-SEG). Inherent biases and errors in the first two post-processing techniques have revealed the need for a robust automated method to derive an accurate noise-free precipitation time series from the raw bucket-weight measurements. The method must be capable of removing random noise, diurnal oscillations, and evaporative (negative) drift from the raw data. This evaluation primarily focuses on cold-season (October to April) accumulating automated weighing precipitation gauge data at 1 min resolution from two sources: a control (pre-processed time series) with added synthetic noise and drift and raw (minimally processed) data from several WMO Solid Precipitation Intercomparison Experiment (SPICE) sites. Evaluation against the control with synthetic noise shows the effectiveness of the NAF-SEG technique, recovering 99 %, 100 %, and 102 % of the control total precipitation for low-, medium-, and high-noise scenarios respectively for the cold-season (October–April) and 97 % of the control total precipitation for all noise scenarios in the warm season (May–September). Among the filters, the fully automated NAF-SEG produced the highest correlation coefficients and lowest root-mean-square error (RMSE) for all synthetic noise levels, with comparable performance to the supervised and manually intensive NAF-S method. Compared to the O15 method in cold-season testing, NAF-SEG shows a lower bias in 37 of 44 real-world test cases, a similar bias in 5 cases, and a higher bias in 2 cases. In warm-season testing, the NAF-SEG bias was lower or similar in 7 of 11 cases. The results indicate that the NAF-SEG post-processing technique provides substantial improvement over current automated techniques, reducing both uncertainty and bias in accumulating-gauge measurements of precipitation, with a 24 h latency. Because it cannot be implemented in real time, we recommend that NAF-SEG be used in combination with a simple real-time filter, such as the O15 or similar filter.

Using the Budyko Framework for Calibrating a Global Hydrological Model

AbstractGlobal hydrological models (GHMs) have become an established tool to simulate water resources worldwide. Most of the GHMs are however uncalibrated and typically use a set of basic hydrological parameters, that could potentially lead to unrealistic projections of the terrestrial water cycle. The calibration of hydrological models is usually performed by using and comparing modeled to observed discharge. Accurate station data and reliable time series data of discharge are, however, often not available for many parts of the world and classic calibration approaches are therefore not feasible. In this paper, we aim to develop a new calibration approach that requires no additional data, is easy to implement, and substantially improves model performance, especially in regions where uncalibrated model performance is rather poor. This is achieved by using the Budyko framework, which provides a conceptual representation of the long‐term water and energy balance. We use a state‐of‐the‐art GHM and calibrate the model within nine river catchments of different sizes and characteristics. Since observed river discharge is available for these catchments, we are able to compare the Budyko‐based calibration approach to a classic discharge‐based calibration scheme and the uncalibrated model version. In all catchments, the Budyko‐based calibration approach decreases biases and increases model performance compared to the uncalibrated model version although performance improvements obtained through a classic calibration approach are greater. Nonetheless, a Budyko‐based calibration is a valuable, intermediate approach between use of a basic set of a priori hydrological parameters and classical calibration against discharge data.

Deep Learning-Based Maximum Temperature Forecasting Assisted with Meta-Learning for Hyperparameter Optimization

Time series forecasting of meteorological variables such as daily temperature has recently drawn considerable attention from researchers to address the limitations of traditional forecasting models. However, a middle-range (e.g., 5–20 days) forecasting is an extremely challenging task to get reliable forecasting results from a dynamical weather model. Nevertheless, it is challenging to develop and select an accurate time-series prediction model because it involves training various distinct models to find the best among them. In addition, selecting an optimum topology for the selected models is important too. The accurate forecasting of maximum temperature plays a vital role in human life as well as many sectors such as agriculture and industry. The increase in temperature will deteriorate the highland urban heat, especially in summer, and have a significant influence on people’s health. We applied meta-learning principles to optimize the deep learning network structure for hyperparameter optimization. In particular, the genetic algorithm (GA) for meta-learning was used to select the optimum architecture for the network used. The dataset was used to train and test three different models, namely the artificial neural network (ANN), recurrent neural network (RNN), and long short-term memory (LSTM). Our results demonstrate that the hybrid model of an LSTM network and GA outperforms other models for the long lead time forecasting. Specifically, LSTM forecasts have superiority over RNN and ANN for 15-day-ahead in summer with the root mean square error (RMSE) value of 2.719 (°C).

Minutely Active Power Forecasting Models Using Neural Networks

Power forecasting is an integral part of the Demand Response design philosophy for power systems, enabling utility companies to understand the electricity consumption patterns of their customers and adjust price signals accordingly, in order to handle load demand more effectively. Since there is an increasing interest in real-time automation and more flexible Demand Response programs that monitor changes in the residential load profiles and reflect them according to changes in energy pricing schemes, high granularity time series forecasting is at the forefront of energy and artificial intelligence research, aimed at developing machine learning models that can produce accurate time series predictions. In this study we compared the baseline performance and structure of different types of neural networks on residential energy data by formulating a suitable supervised learning problem, based on real world data. After training and testing long short-term memory (LSTM) network variants, a convolutional neural network (CNN), and a multi-layer perceptron (MLP), we observed that the latter performed better on the given problem, yielding the lowest mean absolute error and achieving the fastest training time.

Forecasting Cloud Resource Provisioning System using Supervised Machine Learning

One of the biggest challenges cloud computing faces is forecasting correctly the resource use for future demands. Consumption of cloud resources is consistently changing, making it difficult for algorithms to forecast to make precise predictions. Using of the machine learning in cloud computing leads to many benefits. Such as chances of the enhancement in the quality of the service via forecasting future burden of works and responding automatically with dynamic scaling. This motivates the work presented in this paper to predict CPU use of host machines for a single time and multiple times. This paper uses three supervised machine-learning algorithms to classify and predict CPU utilization because of their capability to keep data and predict accurate time series issues. It is tried to forecast CPU usage with better accuracy while comparing to traditional methods.