Learning elastic memory online for fast time series forecasting

Abstract

It is well known that any kind of time series algorithm requires past information to model the inherent temporal relationship between past and future. This temporal dependency (i.e. number of past samples required for a good prediction) is generally addressed by feeding a number of past instances to the model in an empirical manner. Conventional approaches mostly rely on offline model, making them impractical to be adopted in the online or streaming context. Hence, a novel method of online temporality analysis is proposed in this paper. The estimated temporality is then employed to form an Adaptive Temporal Neural Network (ATNN) with an elastic memory capable of automatically selecting number of past samples to be used. Temporality change or drift can be a common occurrence in data streams. Hence a drift detection mechanism is also proposed. Once such drift is detected, a drift handling mechanism kicks in which utilizes the rate of drift, making our solution truly autonomous. The entire mechanism is termed as LEMON: Learning Elastic Memory Online. LEMON although not a time series model in itself, can work with any predictive models to improve their performance. Synthetic datasets are used here as proof of correct temporality estimation and drift detection whereas real world datasets are employed to demonstrate how LEMON improves the predictive performance and speed of an existing model with the knowledge of temporality and drift.