Big Data Time Series Research Articles

Mining and forecasting of big time-series data

Mining and forecasting of big time-series data

An M-estimator for reduced-rank system identification

High-dimensional time-series data from a wide variety of domains, such as neuroscience, are being generated every day. Fitting statistical models to such data, to enable parameter estimation and time-series prediction, is an important computational primitive. Existing methods, however, are unable to cope with the high-dimensional nature of these data, due to both computational and statistical reasons. We mitigate both kinds of issues by proposing an M-estimator for Reduced-rank System IDentification (Mr. Sid). A combination of low-rank approximations, ℓ1 and ℓ2 penalties, and some numerical linear algebra tricks, yields an estimator that is computationally efficient and numerically stable. Simulations and real data examples demonstrate the usefulness of this approach in a variety of problems. In particular, we demonstrate that Mr. Sid can accurately estimate spatial filters, connectivity graphs, and time-courses from native resolution functional magnetic resonance imaging data. Mr. Sid therefore enables big time-series data to be analyzed using standard methods, readying the field for further generalizations including nonlinear and non-Gaussian state-space models.

Designing Multi-Agent Simulation with Big Time Series Data for a Global Supply Chain System

Multinational corporations produce products at relatively few factories and then sell those products in all areas of the world. Longer lead times increase the risk of fluctuations in product demand. To reduce this risk, the entire business chain, from production to sale, must be optimized. In this study, we propose, implement, and verify a total optimization system. This system uses multi-agent simulation on big time series data. It consists of management, integrated database, and operation modules.

Supervised Aggregative Feature Extraction for Big Data Time Series Regression

In many applications, and especially those where batch processes are involved, a target scalar output of interest is often dependent on one or more time series of data. With the exponential growth in data logging in modern industries, such time series are increasingly available for statistical modeling in soft sensing applications. In order to exploit time-series data for predictive modeling, it is necessary to summarize the information they contain as a set of features to use as model regressors. Typically this is done in an unsupervised fashion using simple techniques such as computing statistical moments, principal components or wavelet decompositions, often leading to significant information loss, and hence suboptimal predictive models. In this paper, a functional learning paradigm is exploited in a supervised fashion to derive continuous smooth estimates of time-series data (yielding aggregated local information), while simultaneously estimating a continuous shape function yielding optimal predictions. The proposed supervised aggregative feature extraction (SAFE) methodology can be extended to support nonlinear predictive models by embedding the functional learning framework in a reproducing kernel Hilbert spaces (RKHSs) setting. SAFE has a number of attractive features including closed-form solution and the ability to explicitly incorporate first- and second-order derivative information. Using simulation studies and a practical semiconductor manufacturing case study, we highlight the strengths of the new methodology with respect to standard unsupervised feature extraction approaches.

Dynamic Guaranteed Cost Compression for Time Series Big Data

Query processing technology has recently received a lot of attention in the business intelligence and information service communities. However, the existing approaches can not efficiently optimize the query performance in the uncertain big data environment. In this paper, we propose QPPUBG, a novel and efficient query processing platform for uncertain big data. QPPUBG mainly includes four modules: (i) query equivalence reconstructing for uncertain big data; (ii) multiple query optimization over probability relation components; (iii) query execution plan constructing over probability relation components, and (iv) physical implementation solution of query for uncertain big data. Specially, QPPUBG can support the possible world instance semantics and efficiently handle arbitrary decision spaces. Moreover, QPPUBG can seamlessly integrate the above four modules into the modern parallel computation frameworks. We present the extensive experiments that demonstrate QPPUBG is both efficient and effective.

An Approach for Prediction of Acute Hypotensive Episodes via the Hilbert-Huang Transform and Multiple Genetic Programming Classifier

Acute hypotensive episodes (AHEs) are one of the hemodynamic instabilities with high mortality rate that is frequent among many groups of patients. This study presents a methodology to predict AHE for ICU patients based on big data time series. The experimental data we used is mean arterial pressure (MAP), which is transformed from arterial blood pressure (ABP) data. Then, the Hilbert-Huang transform method was used to calculate patient's MAP time series and some features, which are the bandwidth of the amplitude modulation, the frequency modulation, and the power of intrinsic mode function (IMF), were extracted. Finally, the multiple genetic programming (Multi-GP) is used to build the classification models for detection of AHE. The methodology is applied in the datasets of the 10th PhysioNet and Computers Cardiology Challenge in 2009 and Multiparameter Intelligent Monitoring for Intensive Care (MIMIC-II). We achieve the accuracy of 83.33% in the training set and 91.89% in the testing set of the 2009 challenge's dataset and the 84.13% in the training set and 82.41% in the testing set of the MIMIC-II dataset.

Time Series Analysis for Big Data: Evaluating Bayesian Structural Time Series Using Electricity Prices

In the era of Big Data, time series analysis faces new challenges. The incredible amount of data, which becomes available every day, is accompanied by a similarly huge amount of external variables that could be used to explain or predict data. To cope with those possible covariates, a robust selection of relevant variables is necessary to efficiently predict or explain data. However, a careful distinction between predictive and explanatory analysis is crucial as the two tasks can be tackled differently well by the same method. In this paper, we evaluate new research methods for the analysis of Big Data time series. More specifically, we analyze the performance of Bayesian Structural Time Series and classical machine learning when performing explanatory or predictive tasks. We find that the Bayesian structural time series approach is well suited for the explanatory variable selection, whereas a support vector regression performs better in the prediction task.

Addressing Big Data Time Series

Addressing Big Data Time Series

Addressing Big Data Time Series

Most time series data mining algorithms use similarity search as a core subroutine, and thus the time taken for similarity search is the bottleneck for virtually all time series data mining algorithms, including classification, clustering, motif discovery, anomaly detection, and so on. The difficulty of scaling a search to large datasets explains to a great extent why most academic work on time series data mining has plateaued at considering a few millions of time series objects, while much of industry and science sits on billions of time series objects waiting to be explored. In this work we show that by using a combination of four novel ideas we can search and mine massive time series for the first time. We demonstrate the following unintuitive fact: in large datasets we can exactly search under Dynamic Time Warping (DTW) much more quickly than the current state-of-the-art Euclidean distance search algorithms. We demonstrate our work on the largest set of time series experiments ever attempted. In particular, the largest dataset we consider is larger than the combined size of all of the time series datasets considered in all data mining papers ever published. We explain how our ideas allow us to solve higher-level time series data mining problems such as motif discovery and clustering at scales that would otherwise be untenable. Moreover, we show how our ideas allow us to efficiently support the uniform scaling distance measure, a measure whose utility seems to be underappreciated, but which we demonstrate here. In addition to mining massive datasets with up to one trillion datapoints, we will show that our ideas also have implications for real-time monitoring of data streams, allowing us to handle much faster arrival rates and/or use cheaper and lower powered devices than are currently possible.

Hierarchical structured data logging system for effective lifelog management in ubiquitous environment

The researches for collecting personal daily behaviors and providing lifelog services with them have been recently increasing. Recent advances in mobile devices and sensor technologies have motivated to collect a huge amount of personal lifelog data in real time. With the rapid growth of the need for the research, there is a coming need for the effective lifelog management system which collects time-series big lifelog data sent from sensing devices and extracts major activities through processing them. For the effective lifelog management, the lifelog data can be processed in separated computing resources depending on the size and level of data. In this paper, we propose hierarchical structured data logging to support lifelog based personal services and to reduce the processing complexity and storage cost. First, we present the architecture of personal lifelog management system. With the system we present hierarchical lifelog data logging to optimally utilize computing and storage resources. Then we describe cost analysis and performance comparison for demonstrating the efficacy of our proposed system. Finally, as an initial step for experiments in our research, we describe experimental results of recognizing physical activities and extracting lifelog data which indicate major activities from them.

Real-time analysis and management of big time-series data

The ability to process and analyze large volumes of time-series data is in increasing demand in various domains including health care, finance, energy and utilities, transportation, and cybersecurity. Despite the broad use of time-series data worldwide, the design of a system to easily manage, analyze, and visualize large multidimensional time series, with dimensions on the order of hundreds of thousands, is still a challenging endeavor. This paper describes the Streaming Time-Series Analysis and Management (STAM) system as a solution to this problem. STAM provides the capability to glean actionable information from continuously changing time series with thousands of dimensions, in real time. STAM exploits the IBM InfoSphere® Streams platform and allows for general-purpose large-scale time-series analytics for applications including anomaly detection, modeling, smoothing, forecasting, and tracking. In addition, the system provides user-friendly tools for managing, deploying, and initiating analytics on large-scale data streams of interest, and provides a web-based graphical visualization interface that allows highlighting of events of interest with interactive menus. In this paper, we describe the system and illustrate its use in a large-scale system-monitoring application.